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programs on job retention and unemployment. The training programs 
evaluated by this system will be considered successful in reducing 
unemployment and retaining jobs when they result in the following 
performance indicators: improved worker competencies; improved work 
unit performance; improved competitive standing of the company; and 
stable or improved wages and earnings of trainees. The test of the 
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system in 24 case studies of training programs in California, 
Illinois, and at least one more state. A final report is expected. 
(30 references.) (CML) 



* Reproductions supplied by EDRS are the best that can be made 

* from the original document. 



g STATE-FINANCED, 
g WORKPLACE-BASED 
a RETRAINING PROGRAMS 



U.S. DEPARTMENT OF EDUCATION 

Otfce d Educational Research and improvement 
EDUCATIONAL RESOURCES INFORMATION 
f CENTER IERIC) 

tAhis document has t*en '*P f °^«? " 
Vrece.ved Uom the person or ofQ3n.1at.0n 

onginating it 
D Minor changes ha/e been ntade to .mprove 

reproduction quality 

• Pcnts ol vie* or opm.ons stated -n this .docu- 
ment do not necessarily represent o«.aal 
OERlpoSittonor policy 



A Joint Study of 
the National Commission for Employment Policy 
and the National Governors' Association 



BEST COPY AVAILABLE 



STATE-FSNANCED, WORKPLACE-BASED 
RETRAINING PROGRAMS 



A Joint Study of the 
National Commission for Employment Policy 
and the National Governors' Association 



by 

Peter A. Creticos 

Creticos & Associates, Inc. 
Oak Park, Illinois 

and 

Robert G. Sheets, Ph.D. 

Center for Governmental Studies 
Northern Illinois University 
DeKalb, Illinois 



Research Report # 89-01 
January 1989 



The opinions expressed in this paper are those of the authors and do not necessarily reflect the 
opinions of the members and staff of the National Commission for Employment Policy, the National 
Governors' Association, and the organizations and agencies participating in this study. 



ERJC 



MEMBERS OF THE NATIONAL ADVISO RY COMMITTEE 

John Bishop, Cornell University, Ithaca, NY 

Anthony Carnevale, American Society for Training and Development, 
Alexandria. VA 

Ken Edwards, International Brotherhood of Electrical Workers, Washington, D.C. 

Patricia Buckley Frederick, Machinery & Allied Products Institute, 
Washington, D.C. 

Susan Goguen Herrenbruck, National Center on Occupational Readjustment, 
Washington, D.C. 

Markley Roberts, AFL - CIO, Washington, D.C. 

John Robinson, Motorola, Inc. Schaumberg, IL 

Ralph Smeda, American Bankers' Association, Washington, D.C. 

David Stevens, University of Missouri, Columbia, MO 

Jeff Zornitsky, Abt Associates, Cambridge, MA 



Members ex officio: 

Kristine Coryell, Chief Executive Officer, Prairie State 2000 Authority, 
Chicago, IL 

Steven Duscha, Executive Director, Employment Training Panel, Sacramento, CA 



ERIC 



4 



TABLE OF CONTENTS 



PREFACE 1 

EXECUTIVE SUMMARY 3 

INTRODUCTION 7 

PART A 

I. PROGRAM OBJECTIVES AND ACTIVITIES 11 

H. JOB TRAINING POLICY ISSUES 19 
EI. ECONOMIC DEVELOPMENT POLICY ISSUES 29 
IV. POLICY ISSUES FOR EVALUATION 43 

PARTB 

I. PROGRAM MONITORING AND EVALUATION 49 
n. PROPOSED ADMINISTRATIVE GUIDELINES FOR BUSINESS 

SCREENING 53 
m. PROPOSED PERFORMANCE INDICATORS AND RELATED 

RESEARCH QUESTIONS 59 

IV. NET IMPACT EVALUATION AND RELATED RESEARCH ISSUES 63 

V. PROPOSED RESEARCH DESIGN FOR CASE STUDIES 65 

APPENDICES AND REFERENCES 

APPENDIX A 71 

APPENDIX B 81 

REFERENCES 89 



PREFACE 



Early in 1988, the National Commission for Employment Policy (NCEP) was 
asked by the National Governors' Association (NGA) to support a study of a new 
variety of state training programs. Two states in particular, Illinois and California, 
have pioneered in the development of these "state-financed, workplace-based" 
programs that attempt to aid firms and their employees simuitaneously. 

Federally-sponsored programs targeted on experienced workers have tended 
to intervene only after the workers have been "dislocated", and the employment 
connection with their old firm has been broken. Part of the difference between the 
dislocated worker approach and the approach studied in this project is that 
forestalling unemployment and the social costs that go with it is an explicit priority. 
Another part of the difference is that firms aided under these programs are typically 
expected to develop a plan for improved competitiveness of which retraining is but 
one dimension. 

In addition to financial support, Commission policy analysts Kay Albright 
and Stephen Baldwin have been substantively involved in the development of the 
project and the editing of this report. A similar contribution has been made by 
Evelyn Ganzglass, director of the NGA's employment and training program. Both 
organizations are grateful to the authors of the report, Peter Creticos and Robert 
Sheets, and to the members of the project advisory panel, whose names aie listed 
elsewhere in this report. Particular attention also goes to the board members and 
staffs of the state programs that are being studied, and to the officials and 
employees of the firms that have cooperated with the researchers. 

NCEP and NGA expect to publish the final report on the project in early 
1990, which will contain the results of the case studies to be conducted within the 
present framework. 
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EXECUTIVE SUMMAR V 



Increased national and international competition, rapidly changing 
technologies, chronic problems of worker dislocations and lagging growth in 
productivity have brought together American businesses, unions and government in 
new public-private partnerships for retraining experienced workers. State-financed, 
workplace-based employee retraining programs have been established recently as a 
part of state efforts to improve the competitiveness of firms operating in the U.S. 
and to prevent unemployment by retraining employees who are at risk of losing 
their jobs because of changing skill requirements in the workplace. California's 
Employment Training Panel and Illinois' Prairie State 2000 Authority are two state 
agencies that partially underwrite retraining costs. These agencies provide funds to 
employers to retrain those employees whose skills have been made obsolete by 
changes in process technologies and work organization. 

These new state-financed, workplace-based retraining programs represent a 
major departure from earlier government employment and training policies and 
programs. Previous government programs have been rooted in broadly accepted 
policy premises that justify the need for government involvement ; n universal 
education and pre-employment training for new labor entrants, the educationally 
and economically disadvantaged and displaced workers. 

Until recently, job-related training for privately employed workers was 
viewed simply as a normal cost of doing business and as an activity within the 
exclusive domain of the private sector. This policy perspective was based on 
premises that the benefits from job-related training accrue primarily to the 
employer sponsoring the training and that the workers' newly acquired skills 
generally are not transferable to other employers. This perspective has restricted 
government investment to general skill training outside the workplace. 

These new state-financed, workplace-based retraining programs also are 
major components of new state economic development efforts aimed at retaining 
jobs by improving the competitive standing of businesses in their own industries. 
These efforts include technology transfer assistance, marketing and export 
assistance apd financing assistance, like other state business retention programs, 
these new state-financed, workplace-based retraining programs attempt to improve 
company performance in areas (e.g., unit costs, quality control, delivery time) that 
are critical for businesses to remain competitive within their own industries. 

These retraining programs serve as a bridge between employment and 
training and economic development programs because they establish the 
importance of integrating human resource investment with capital expenditures on 
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new process technologies. They also direct state attention toward coordinating job 
training and economic development programs in efforts to improve the competitive 
standing of businesses within the state. 

Major structural changes in the international and national economies and 
growing concerns about the competitiveness of American businesses hi/e prompted 
public policy-makers to question the adequacy of government involvement in 
general sldll training and consider new government involvement in workplace-based 
employee retraining. Some states recently began financing worker retraining in the 
workplace in the belief that retraining programs are an efficient and effective means 
of keeping skilled workers in productive jobs and achieving improvements in 
business competitiveness. The emerging policy rationale for these new state 
programs is based on the argument that job-related training of employees involves a 
wide range of transferable basic and vocational skills that prevent companies from 
capturing the full economic benefits for this type of investment. This is true 
especially for the advanced process technologies and new forms of work 
organization that businesses are attempting to implement to remain competitive. 
Employee retraining for these types of changes requires a large degree of general 
skills training that easily can be lost if workers get hired away by other companies 
who did not incur the training costs. 

In addition, this emerging policy rationale argues that government has a 
direct interest in retraining employees to prevent or reduce the social and economic 
costs of technology and economic change. In particular, government has an interest 
in preventing unemployment and keeping workers off unemployment insurance. 
Government also has a direct interest in encouraging companies to respond to 
technology and market pressures by retraining employees, especially non- 
managerial and non-professional employees who traditionally have received less 
government and business training investment. These programs also are intended to 
encourage further training investment by businesses that extend beyond the period 
of government investment. 

Growing business interest in state-financed, workplace based retraining 
programs and the resulting demands for more public funds have prompted public 
debate on the proper role of these state-financed, workplace based retraining 
programs in state employment and training and economic development efforts. This 
debate also concerns the effects of these programs on reducing unemployment and 
retaining jobs. This research and evaluation project represents the first effort to 
develop and test a comprehensive monitoring and evaluation system tailored 
especially to the objectives and admmistritive requirements of state-financed, 
workplace-based employee retraining programs. 
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A comprehensive monitoring and evaluation system for state-financed, 
workplace-based retraining programs should have three major components: 

First, a process monitoring system that establishes whether these programs 
are following administrative guidelines for program targeting and 
substitution, 

Second, a performance assessment system that measures direct program 
outcomes and establishes whether these programs are achieving the 
results at the work unit and company levels that are necessary for 
having significant impacts on unemployment and job retention, and 

Third, an impact evaluation system for measuring directly the net impact of 
these programs on unemployment and job retention. 

In developing a monitoring and evaluation system for these state programs, 
this project will concentrate on the development of a process monitoring and 
performance assessment system that can establish whether these programs are 
following administrative guidelines for program targeting and substitution and 
whether they are achieving the program outcomes that are necessary for achieving 
significant impacts on unemployment and job retention. This project also will 
propose alternative net impact designs that state agencies could consider in 
developing an impact evaluation system for these programs. 

Based on the intervention models of California's Employment Training 
Panel and Illinois' Prairie State 2000 Authority, this project has designated four 
major program outcomes as the foundation of a performance assessment system: 

o Certified Competencies in Transferable Skills 

o Improved Work Unit Performance 

o Company Improvement in Competitive Standing 

o Wages and Earnings of Trainees 

State-financed workplace-based retraining programs would be considered 
successful in reducing unemployment and retaining jobs when training projects 
result in improved worker competencies, improved work unit performance, 
improved competitive standing of the company, and stable or improved wages and 
earnings of trainees. Some state agencies already collect information on wages and 
earnings of trainees as part of their current evaluation efforts. However, no state 
agency evaluates training projects in terms of improved business performance. 
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The project case studies will focus predominantly on an assessment of the 
appropriateness and feasibility of the four performance outcome indicators 
proposed in this paper. The project case studies will be conducted in two major 
phases. The first phase will review all training projects completed during the two 
previous calendar years by California's Employment Training Panel and Illinois' 
Praine State 2000 Authority ETAP program This phase will include exploratory 
case studies on 3 training projects. The second phase of the project will involve an 
addmonal 21 case studies that will apply the proposed set of performance outcome 
measures m an evaluation of training projects in California, Illinois and at least one 
additional state. These case studies will attempt to measure program outcomes on 
all proposed indicators where data are available. These project evaluations will 
result m a final report assessing the feasibility of the proposed evaluation system 
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INTRODUCTION 



In recent years, state governments have established new and innovative 
workplace-based retraining programs centered on the principle that the most 
effective way to address unemployment is to prevent it in the first place. These 
programs are operated with the mandate to minimize the incidence of 
unemployment and retain jobs by retraining employees Who are at risk of losing 
their jobs because of changing skill requirements in the workplace. State agencies, 
such as California's Employment Training Panel and Illinois' Prairie State 2000 
Authority, provide retraining funds to companies for workers whose skills no longer 
match the requirements of their jobs due to changes in process technologies and 
work organization used in the production of goods and services. 

Growing business interest in these programs and the resulting demands for 
more public funds have raised important policy questions about the proper roles of 
state-financed, workplace-based retraining in job training and economic 
development efforts: Inasmuch as these programs have been experimental, most 
state policy-makers have been satisfied thus far with the general argument that 
retraining will lead to reduced risks of unemployment and to job retention. 
However, critics contend that a sufficient case has not been made for this kind of 
public sector participation in the operations of private businesses. They argue that 
retraining is the responsibility of business and that government assistance does no 
more than substitute public resources for private ones. Also, they argue that there is 
insufficient evidence to support the claim that worker retraining is an effective 
strategy for reducing unemployment and retaining jobs. It is reasonable to expect 
that as state-financed, workplace-based retraining programs become more common, 
these criticisms will be accorded detailed consideration by policy makers. 

Although the policy rationale for many of these programs continues to 
evolve, it appears that state-financed, workplace-based retraining offers a 
convenient means for linking the goals of employment and training policies with 
those of economic development. Employment and training policy advocates hold 
the view that training programs achieve reductions in unemployment and retention 
of jobs b'' improving the skills of workers. On the other hand, economic 
development policy advocates view state-financed, workplace-based retraining a 
means for achieving lasting improvements in the ability of a firm to compete 
effectively within its given industry. 

Programs like those offered in California and Illinois appear to link 
employment and training pol'cies with economic development policies because they 
provide state retraining assistance to businesses that show that: 1) improvements are 
being made in process technologies and work organization used in the production of 
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goods and services, and 2) worker skill requirements have changed. Although this is 
done mainly to document the need for retraining, one outcome is that retraining is 
tied to broader company strategies for enhancing or gaining a competitive position. 
As a consequence, boih employment and training and economic development 
objectives seem to be achieved because the combination. of improvements in worker 
skills and company performance presumedly will result in greater prosperity for the 
company and a reduction in the risk of unemployment for the newly retrained 
employees. 



Agencies operating these state-financed, workplace-based retraining 
programs, m particular Illinois' Prairie State 2000 Authority and California's 
Employment Training Panel, have worked to clarify the criteria for targeting public 
subsidies and have developed some indicators of program performance For 
instance, they have borrowed evaluation methods established for other government- 
sponsored employment and training programs. Initial program performance 
evaluations have included assessments of the effects of worker retraining on trainee 
skills and pre- and post-training comparisons of employment and earnings 
However, these agencies recognize that they require a more fully developed 
evaluation system if they expect to target their resources or assess their effect on 
unemployment and job retention. 

The first step in the development of a comprehensive evaluation system for 
state-financed, workplace-based retraining programs is to establish a clear policy 
rationale for government intervention and a formal intervention model that 
specifies the program outcomes that are necessary to achieve the desired effect on 
unemployment and job retention. The policy rationale and intervention model for 
these programs are discussed in Part A of this report. 

The first section of Part A reviews competing perspectives on the role of 
government m job retraining for employed workers and summarizes the emerging 
policy rationale for state-financed, workplace-based retraining programs in the 
context of employment and training policy issues. This section is followed by a brief 
discussion of economic development policy issues and the problems of targeting 
government funds and minimizing substitution effects. Part A concludes with a 
summation of the intervention model and of the targeting and substitution issues, 
rnese will provide the foundation for a monitoring and evaluation system. 

Part B of this report addresses the major components of a monitoring and 
evaluation system for state-financed, workplace-based retraining programs and 
establishes the program outcome indicators that will be assessed in project case 
studies. v J 
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The first section of Part B reviews the major components of an evaluation 
system. The second section describes proposed administrative guidelines for 
business selection and substitution minimization. The third section describes the 
four performance outcome indicators for a performance assessment system. The 
reniaining sections of Part B describe the research design i id key research issues 
for the project case studies. 
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I. 



PART A; POLICY ISSUES 
PROG AM OBJECTIVES AND ACTIVITIES 



Increased national and international competition, rapidly changing 
technologies, chronic problems of worker dislocation, and lagging growth in 
productivity have brought together American businesses, unions and government in 
new public-private partnerships for retraining experienced workers. At least a 
quarter of the states have responded with financial assistance to businesses that 
covers a portion of the cost of retraining workers. These programs differ markedly 
from other publicly-financed, workplace-based training programs that routinely 
subsidize the cost of training new employees as components of state financial 
incentive packages to attract new businesses or encourage business expansion. They 
differ also from programs that are intended to solve the problems of dislocated 
workers in that they attempt to address the problem of dislocation before it occurs. 
The result are programs that seek to retain existing jobs by improving the 
competitiveness of resident businesses largely by combining employee retraining 
and improvements in individual productivity with technological innovation in the 
workplace. They also are intended to encourage businesses to adopt more 
aggressive human resource investment strategies that provide for continual 
retraining of their w^ers. 

Although many states have programs that occasionally provide financial 
assistance to businesses for retraining workers, two states, Illinois and California, 
have ongoing, long-term programs that tie the use of retraining funds with 
concurrent changes in the workplace. Although born under different circumstances, 
these programs share essential common traits that separate them from other 
industry-specific training programs. 

California. 

In 1982, the California legislature established the Employment Training 
Panel to: 

foster job creation, minimize employers' unemployment 
costs, and meet employers' needs for skilled workers by 
providing skills training to unemployment insurance 
claimants, recent exhaustees of unemployment insurance 
who have remained unemployed, and potentially 
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displaced workers who would otherwise become 
unemployment insurance claimants. 1 



The Panel is composed of seven members with experience and a 
demonstrated interest in business management and employment relations. Four 
members of the panel are appointed by the Legislature: two by the Speaker of the 
Assembly and two by the President pro Tempore of the Senate. Three members are 
appointed by the Governor. The Governor also designates the chair from among 
the members. The Panel hires the 3cecutive Director, who serves at its pleasure, 
and stan; who are subject to the State Civil Service Act. Two Assistant Directors 
appointed by the Governor, also serve at the pleasure of the Panel. 

Funds appropriated to the Panel may be expended only to help people learn 
skills that will put them to work or to help people learn skills that will keep them 
from losing their jobs. Money from the Fund are spent "only for training for 
employers or groups of employers who assure that those who successfully complete 
training will be employed " In addition, the Panel must be convinced that the training 
will move people into careers with long-term job security. It will not, as a matter of 
policy, train people for "deadend, minimum wage jobs... [or]. ..short term or high 
turnover jobs. " Its objective is to "help train people for good jobs that provide a decent 
living in stable or expanding occupations." The Panel writes performance-based 
contracts that tie payment to the successful completion of training and 90-day post- 
training retention It will reimburse the employer for all actual training costs and 
reasonable administrative expenses. However, it will not subsidize wages paid to 
trainees nor pay stipends. In the event that the workers receiving training are union 
members, the Panel requires that the union must first agree to the training before 
state funds are expended. 

Funding for the panel is derived from receipts from a tax imposed on 
employers. The basis for the tax is identical to California's unemployment insurance 
tax. Annual appropriations to the Panel by the Legislature have grown from $26 
million in fiscal year 1983 to amounts in excess of $60 million in fiscal years 1986 
through 1988. However total revenues regularly have exceeded their annual 
appropriations. For example, in FY 1988, total revenues exceed $104 million, 
including money carried" forward from the previous year and estimated 
disencumberances. 

Initial Panel activities were focused on projects that trained unemployed 
workers. However, since its inception through June 30, 1987, 53 percent of the 
projects have trained potentially displaced workers. Another 20 percent of the 
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projects have combined unemployed and potentially displaced workers. The 
remaining 27 percent of the projects trained only unemployed workers. 



Illinois. 

The purpose of the Prairie State 2000 Authority is to: 

establish € ^vloyment training programs which foster job 
creation, reauce employer unemployment costs, and meet 
the needs of the economy for skilled workers by providing 
job-linked training for unemployment insurance claimants 
and potentially displaced workers who could become such 
claimants. 2 

It was established in 1983 under a slightly different name initially to operate 
a system of individual training accounts. In 1985, the mission of the Authority was 
revised and two programs were added. One program, the Individual Training 
Assistance Program, was established to provide financial assistance to experienced 
Ul-eligible workers who wanted to upgrade their skills or acquire new ones. This 
program was made available to workers who were unemployed as well as those who 
still were working. The second program, the Employer Training Assistance 
Program, was established "to make grants or loans to eligible employers for the 
purposes of providing training to employees in fields for which there are critical 
demands for certain skills." It also provides that the agency provide financial 
assistance to an employer: 

1) who will provide job-linked training which 
offers special skills for career advancement or which is 
preparatory for, and leads directly to, jobs with definite 
career potential and long-term job security; 

2) who is unable to provide sufficient funds 
internally, or from other available sources, including 
federal, State or locally administered employment and 
training programs; and 

3) (i) who is expanding its business enterprise in 
this State, is locating a new business enterprise in this 
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State, is introducing more efficient technology into its 
operations which will result in greater output per 
employee, is expanding into new markets, or is expanding 
exports from Illinois, and is thereby increasing tax revenues 
for State and local governments; or 

ii) whose existing employees are threatened 
with layoff unless additional training is made available to 
them. 

The Authority is governed by a seven-member Board. Four members are 
appointed by the Governor with no more than two from the same political parly. 
The other three members are ex officio: the State Treasurer, the Director of the 
Department of Commerce and Community Affairs, and the Director of the 
Department of Employment Security. The ex officio members may designate others 
to attend meetings of the Board in their place. The Board elects a Chair from 
among the four appointed by the Governor. It also appoints the Chief Executive 
Officer, who serves at its pleasure, and the staff. 

In implementing the statute, the Authority has used the Employer Training 
Assistance Program to assist employers retrain their workers in new process 
technologies or in new forms of work organization. Pursuant to the statute, the 
Authority requires that the employer demonstrate that it lacks the financial 
resources to conduct the retraining in a proper or timely fashion. Finns or 
establishments that are eligible for assistance include those earning little or no 
profits, or those reinvesting their earnings back into their operations. In addition, as 
a condition of assistance, the Authority requires that the firm or establishment 
making the request provide a reasonable business strategy that incorporates the 
proposed retraining. like the California program, if the workers receiving the 
retraining are union members, the Authority requires that the union first agree to 
the retraining before the grant or loan request may be approved. The agency will 
give grants covering 50 percent of the direct training costs or low-interest loans 
covering all eligible costs. 

Agency performance since fiscal year 1986 shows considerable year-to-year 
consistency. In fiscal year 1986, the Authority issued 48 gnats that resulted in 
retraining for nearly 4,000 workers. Additional grants were made to a major Illinois 
manufacturer under an experimental program that resulted in another 13,000 
workers receiving retraining. In total, over $1 million was spent for employer 
training assistance. In FY 1987, the agency made 61 grant awards for over $937,000 
and 7 loans for over $54,000. These trained 8500 employees. In FY 1988, the 
agency made 56 grants ($927,000) and g loans ($58,000) covering 9735 workers. ' 
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Observations, 



The agencies operating California's Employment Training Program and the 
Illinois' Employer Training Assistance Program, although differing in size and some 
operating procedures, share key characteristics. For instance, each agency is 
required by law to serve unemployed workers. Also, both are charged with the 
mission of preventing unemployment by assisting the employed to learn skills that 
are in demand. 

In looking to lower the risk of unemployment when jobs are in jeopardy, the 
statutes of California and Illinois require contractor employers to continue 
employing workers that were retrained with state assistance for at least ninety days. 
This provision is implemented in California through what the Employment Training 
Panel calls "performance-based" contracts. It pays only the training costs for 
workers who remain on the job ninety days after the completion of the retraining 
program. In light of cash flow constraints felt by the employer, the Panel may make 
progress payments not exceeding 50 percent of the total grant contract. Illinois 
takes a slightly different approach in that it routinely pays one half of the grant at 
the time training commences. The other half of the grant, is prorated and paid on 
the basis of the number of workers who remain on the job 90 days after completion 
of the training program. Loans are awarded entirely at the front end of the training 
activities. However, up to 25 percent of the principal may be forgiven on the basis 
of the share of newly retrained workers remaining on the job one year after the 
completion of training. 

In addition, California practice and Illinois statutes require that the applicant 
business describe the company's long-term business strategy and the role that 
retraining will take in implementing this business strategy. One consequence of 
these inquiries is that the agencies appear to screen out applicants that do not 
provide a productive work environment. The assumption implied by such an 
approach is that retraining is not in itself sufficient to achieve significant operating 
improvements leading to a reduction in the risk of unemployment. Other factors in 
the work environment affect the performance of the worker. These factors include 
the obvious, such as the business providing its workers with the appropriate 
machinery and equipment that are in good working order. Or they may include 
more subtle environmental changes. For instance, statistical process control (SPC) 
systems adopted at the plant floor generally must be accompanied by other changes, 
such as tighter controls on raw materials and parts, in order for there to be a 
significant improvement in the quality of the final p r Dduct. This requires that 
material buyers adopt new techniques in order for tht training in SPC to have a 
noticeable effect. 
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Although the fundamental objectives of the two programs are very similar, 
there are some differences. Perhaps the most critical difference is in funding 
source. California's program is operated with money received through an 
unemployment-insurance type tax on employers paid in behalf of workers. The 
primary consequence of this approach is that the business community is viewed as 
taking an "ownership" stake in the program: essentially, business money is collected 
and redistributed to help a subset of businesses making investments in human 
resources. This contrasts with the Illinois approach that takes a more traditional 
government position: i.e., general funds are used to achieve specific social and 
economic objectives. 

These differences are manifested in one key way. The Employment Training 
Panel is not obligated to give priority to or specifically serve financially needy 
businesses. Instead, the emphasis is on performance, particularly retention. This 
appears to be a direct means of achieving the fundamental objective of lowering the 
incidence of unemployment. On the other hand, Illinois gives a great deal of 
attention to financial need, primarily because it does not want to substitute 
government funds for money that the company otherwise would have spent for 
retraining. 

While it is possible that the source of funds may have an effect on 
government policies as they pertain to substitution, it is not unusual for a state 
government to decide on a program and then seek the most expedient means to 
fund it. For instance, until recently, Illinois' business community has had to live with 
unusually high unemployment insurance tax rates. As a result, there was no capacity 
in the unemployment insurance system to support any programs, particularly 
experimental ones like the Employer Training Assistance Program. On the other 
hand, California had substantial capacity in the unemployment insurance system by 
virtue of the tax cuts being made at the time the Employment Training Panel was 
established. Consequently, employers still received their tax cut. The only 
difference was that the cut was not as great as it otherwise would have been had not 
the Employment Training Panel been established. 

However, setting aside the argument that expediency may be a major reason 
for the differences in funding approach, the examples of California and Illinois 
suggest that differences in the source of funds do not necessarily result in program 
differences. For instance, the California Panel views the source of its funds as 
coming from the business community although paid in behalf of workers. Labor 
economists generally argue that while the unemployment insurance program, and 
therefore the Employment Training" Panel program, may be funded directly by 
business, labor contributes directly to the program in the form of lower wages. On 
the other hand, the Illinois General Assembly thought it appropriate to apply 
general funds for essentially private purposes. Many public policy theorists 
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generally object to the use of government money as the source of business subsidies. 
Nevertheless, irrespective of the source of funds, the two programs are remarkedly 
alike. 

Having gone beyond the traditional objectives of employment and training 
policy and into matters pertaining to the long-term viability of the firm, the 
Employment Training Program and the Employer Training Assistance Program 
have crosses into areas traditionally considered to be economic development policy. 
Not only are the agencies that operate these programs concerned about the status of 
the individual receiving the training, they involve, themselves also in the condition 
and operational improvements of the firm. They have, in effect, established a 
hybrid approach that suggests that positive changes in competitiveness are a 
function of worker productivity improvements and workplace changes. 
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H. JOB TRAINING POLICY ISSUES 



The history of government policy in the education and training of workers for 
private employment describes the growing intervention of the public sector in labor 
markets. One purpose of this intervention has been to reduce the mismatch 
between the skills of workers and changing workplace requirements resulting from 
major technological and economic shifts. 

Most of the public sector's efforts in worker education and training have 
been contained within three major areas, each having been established to address 
specific labor market problems brought to national attention during particular 
historical periods. These three areas of government-financed worker education and 
training are: (1) universal education and pre-employment vocational training for 
new labor market entrants; (2) pre-employment education and training for 
disadvantaged workers, and (3) retraining assistance for unemployed and dislocated 
workers. National interstate labor exchange programs add another important 
dimension to the assistance provided to unemployed and dislocated workers. 

The first area of government efforts in education and training stems from the 
public's acceptance of and general commitment to universal primary and secondary 
education and pre-employment vocational training. This commitment was gained, in 
part, because new labor market entrants, mostly school-aged youth, needed to be 
trained in basic and vocational skills necessary for factory and office jobs emerging 
during the 19th century from the new industrial economy. However, over the last 50 
years, pre-employment training has grown substantially beyond the traditional 
primary and secondary education systems. During this time, government has added 
wide arrays of one- and two-year post-secondary vocational education programs and 
comprehensive community colleges providing customized training services to local 
businesses. 

The second area was established by the social programs of the 1960s. It used 
education and training programs as a means for reducing social welfare dependence 
and providing equal education and employment opportunities for economically and 
educationally disadvantaged populations. This, in part, was justified on equity 
grounds and as a less costly alternative to the social welfare costs of mamtaining a 
dependent, non-productive population at the margins of the labor market. Efforts 
along these lines continue to grow through a variety of federal and state programs, 
including the Job Training Partnership Act and new welfare-work programs. 

The third area of government commitment to education and training is 
characterized by a rapidly growing number of new federal and state programs that 
retrain non-agricultural unemployed workers who lose their jobs because of major 



19 



technological and economic changes. These programs emerged during the 1960s with 
the Economic Redevelopment Act and the Manpower Development and Training 
Act in response to national concerns over the displacement effects of technology 
changes in the wcAplace. It was strengthened during the 1980s with the Title m 
JTPA Dislocated Worker*, and the revised Trade Adjustment Assistance (TAA) 
programs in response to pac-lic concerns over manufacturing job losses due to 
structural changes in the national economy and foreign competition. A more 
comp-ehensive national program Los been enacted into law that is part of the 1988 
Omnibus Trade Act and related legislation. In addition, a number of states have 
established their own dislocated workex programs [Ganzglass and Heidkamp, 1986]. 
Some states also have taken the step of enacting advanced notice or other forms of 
plant closing laws [Leroy, 1983]. Generally, tiiese programs have been justified by 
the need to reduce the social and economic costs of economic change by shortening 
the period of unemployment faced by dislocated workers. 

As noted previously, many state workplace-based retraining programs, 
especially in Illinois and California, were established originally to retrain 
unemployed workers to meet the demands imposed by rapidly changing skill 
requirements in businesses. Their transformation into workplace-based retraining 
programs for employed workers may be justified as a more equitable and efficient 
means for matching workers' skills and changing job requirements. If this rationale 
gains broad acceptance, publicly-financed, workplace-based retraining of employed 
workers will become a fourth area of government-financed worker education and 
training. 

However, broad acceptance of these programs is not necessarily a foregone 
conclusion. Most critics of this fourth area rely on traditional policy arguments 
regarding the respective roles of government and business in employee training. 
These arguments, in turn, are based on distinctions between general and firm- 
specific training and assumptions about who actually captures the benefits of 
employee training in the workplace. 



Governmen t Financing of Employee Education and Tr aining. 

Before presenting the policy rationale for government financing of job- 
related retraining for employed workers, it is useful to distinguish job-related 
retraining from other types of employee training. Doeringer [1981] has 
distinguished three general types of employee training in the context of government- 
funded programs: 
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1) Remedial Education and Training: Education and training that 
compensates for short-comings in the public education and pre- 
employment training system or assists in improving the economic 
situation of socially, educationally or economically disadvantaged 
workers; 

2) Employee/Career Development: Job training for a particular job 
or occupation (e,g., certification, required degrees) within the 
company as well as more general educational and employee 
development activities that initially benefit the employee more than 
the company; and 

3) Job-Related Training: Job training that builds workers skills for 
specific workplace applications within a particular compa^v or 
industry and is designed to improve company performance and worker 
productivity. 

Remedial education and training has been justified on the grounds that the 
general social and legal responsibilities of government are to improve economic 
opportunities for disadvantaged workers* This improvement is accomplished by 
means of indirect grants through public education and training organizations for 
upgrading basic and vocational skills, as well as direct wage subsidies for on-the-job 
training and work experience. Despite an apparent upsurge of business 
participation in remedial education and training, the major financing responsibility 
remains with government. Examples of these programs include JTPA wage 
subsidies for on-the-job training and work experience, the Targeted Jobs Tax Credit 
(TJTC) program and workplace literacy programs. 

Employee/career development training supports the development of general 
vocational skills rather than job-specific skills tied to a particular employer. The 
goals of these programs are to bring individual workers up to a minimum level of 
competency and to provide the business community with an adequate labor supply. 
Although individuals often bear the total direct costs of employee/career 
development training, it is common for this training to be financed through cost- 
sharing arrangements involving employees, employers, and government. Individuals 
commonly are expected to share in the costs of his type of training because they 
have an assumed responsibility to obtain training that is necessary to establish and 
maintain a career. These financing mechanisms may include direct employee 
contributions or may be part of an employer-sponsored fringe benefits package (e.g., 
tuition assistance programs, professional conferences, workshops and training 
courses and in-house training programs). 



Government subsidies of this training also come in many forms. Indirect 
subsidies include special deductions or tax credits for certain employer-sponsored 
tuition aid plans. Direct subsidies include student loans, Cooperative education 
programs and government funding for off-campus adult and continuing education. 
Irrespective of the form that these subsidies may take, government funding is 
justified in terms of various social values, such as improvements in human capital, 
support of general business development and equal employment opportunity. 

In contrast, job-related training traditionally has been viewed by employers 
and government as a normal cost of doing business. Consequently, companies have 
had primary responsibility for conducting such employee training as is necessary for 
them to be competitive in their own industries. This attitude has been based 
generally on two assumptions: 1) work skills acquired through job-related training 
usually are not transferable directly to other employers; 2) the employer who 
sponsors the training captures the major portion of the economic return on training 
through improved productivity and firm profitability. From this viewpoint, 
government should share only in the costs of general skill development and should 
encourage employers to invest in job-related training where they are most likely to 
capture the direct economic benefits. 

Policy Rat ionale for Public Financing of lob-Related Retraining for Employees. 

Wrenching changes in the international economy and concerns regarding the 
competitiveness and productivity of Ameiican businesses in most major industries 
have prompted some public officials and policy analysts to propose a more active 
financial role for government in support of workplace-based retraining programs. 
The emerging policy rationale rests on a number of efficiency and equity arguments. 

The efficiency arguments are based on three major issues: 

o externalities in job-related retraining; 

o the social cost of worker displacement; and 

o cost-effective retraining strategies and market incentives for labor 
market intermediaries. 

Those who argue from an efficiency standpoint focus on the wide range of 
transferable basic and vocational skills that are gained in job-related retraining and 
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on the complex externalities that companies face in capturing the full economic 
benefits of this type of investment. Except for a narrow range of training programs 
(e.g., vendor training, company orientation classes, and safety training), many skills 
gained in workplace-based retraining may be demanded by and transferred to other 
employers within the same industry. A great deal of workplace-based retraining: 1) 
upgrades computational and communication skills of workers; 2) provides them 
with practical knowledge of the application of technology in the workplace; and 3) 
improves their adaptability and willingness to learn new job tasks and work 
processes. Also, employers within the same industry often operate equipment of 
5 similar design. Experience on one machine frequently serves as a valuable 

reference point for learning the specific skills in operating other machines involved 
in similar production processes. 

One outcome stemming from the capacity of workers to transfer their skills 
from one job to the next is that the company making a retraining investment many 
.times is unable to achieve an adequate return on this investment. The return on 
investments in retraining often is achieved over several years. However, 
competitors may benefit sooner because they may bid away workers retrained in 
new technologies without incurring any of the costs This free-rider problem will 
develop especially when there is an emerging demand tor workers with particular 
skills. In light of high turnover rates in most labor markets, businesses wili tend to 
under-invest in employee retraining. This likely will result in losses in productivity 
and competitiveness. Government's participation mitigates this free-rider dilemma 
by socializing some of the costs of retraining, thereby lowering the total exposure by 
the business providing the retraining. 

Faced with these problems, businesses often respond to rapid technological 
changes and market shifts by laying off employees with obsolete skills and hiring 
other workers as opposed to retraining employees to meet new job requirements. 
Although this practice may limit direct operating expenses for individual businesses, 
it results in large social welfare costs to government and unemployment insurance 
costs to the business community as a whole (particularly in light of the fact that no 
unemployment insurance r'stem is fully experienced-rated). An alternative is to 
publicly subsidize retraining in the workplace rather than retrain unemployed 
workers for new jobs. The objectives of this approach are to improve aggregate 
economic efficiency and to lessen the social costs of economic change. 

Proponents of direct employer subsidies also maintain that employers 
provide the most cost-effective mechanism for organizing and delivering retraining, 
especially t( those workers at the lower tiers of the labor market. They argue that 
basic and vocational skill training is accomplished most efficiently in the workplace 
where there is a clear connection between general skill training and specific job 
requirements. In addition, workplace-based retraining provides an alternative to 
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adults who were unsuccessful in secondary or post-secondary schools, but who have 
been successful in adopting good work skills. Finally, the shift of government 
financing from educational institutions and other labor market intermediaries to 
employers increases market pressures on public ironing providers to be more 
sensitive to changing skill requirements in the workplace and be more open to 
adapting new educational technologies to improve delivery of adult training. [Office 
of Technology Assessment, 1988; Perelman, 1985] 

The equity argument targets imbalances in government training subsidies in 
the workplace and the need to direct more subsidies to lower tiers of the labor 
force. Government and private investment in education and training generally favor 
managerial and professional people over blue-collar, clerical and other kinds of 
workers in the United States [U.S. Department of Labor, 1985, Office of 
Technology Assessment, 1986]. Many proponents of direct government support of 
employee training argue that as a matter of equity, government financial assistance 
should be extended to lower tiers of the labor force. 

Young people who attend college often are able to pyramid assistance from 
public and private sectors throughout their working lives with Tairly certain returns, 
such as higher earnings and stable employment. Historically, employers and 
government have relied predominantly on an informal system of on-the-job training 
for the remainder of the workforce, including most production workers. Yet, for 
college-educated managers and professional, public subsidies of higher education 
are complemented by formal, employer-financed training programs and tuition aid 
plans. This phenomenon results in a pattern of investment that is biased in favor of 
the upper tiers of the corporate workforce as well as r favor of higher educational 
institutions. Such a bias is exacerbated further by the relatively high levels of 
disposable income of workers in these upper tiers that may be applied to advanced 
education and training programs. 

These efficiency and equity arguments merge when considering productivity 
and competitiveness problems in the United States. Lester Thurow has observed 
that significant overall improvements in competitiveness and productivity cannot be 
achieved by concentrating public and private investment in only college graduates 
who have entered managerial and professional positions [Thurow, 1985]. 

New challenges presented by rapid technological change in the workplace 
provide major reasons for re-examining the public role in financing employee 
retraining. Most technical skill training follows predictable cycles [Flynn, 1988] 
with the burden of training in the early stages of a new technology falling 
predominantly on businesses. Public involvement develops in later stages as n&w 
technologies become more widely disseminated. However, changes in technology 
and foreign competition have forced additional costs onto businesses. It seems that 
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with evexy new day, the technical frontiers in manufacturing and office automation 
are pushed out. As a consequence, many businesses have been required to establish 
formal in-house training programs for non-managerial ana .on-professional 
employees in order to remain competitive. 

Recent studies estimate that the "occupational half-life", the time it takes for 
one-half of workers' skills to become obsolete, has declined from 7-14 years to 3-5 
years [National Research Council, 1986]. Some companies report that the "half-life" 
may be as short as one year. If this trend continues, the public sector may need to 
change its emphasis from directly providing skill training and instead concentrate 
more on teaching workers the underlying concepts and general application of new 
workplace technologies. Otherwise, the public sector will fall increasingly behind in 
meeting private sector demand for workers with the appropriate education and 
skills. In either event, businesses will have to absorb a greater share of the training 
burden in general skills. Unfortunately, in an effort to suryive and without public 
subsidies, businesses may find it necessary to lay-off marginal workers instead of 
retrain them. The result will be higher levels of transfer payments, increased 
demand for various social and public health services and greater pressure on 
targeted employment and training programs serving economically disadvantaged 
and dislocated workers. In addition, these technology shifts and other structural 
changes in the economy will force workers to change jobs and occupations many 
times over their working lives, making continuing education and retraining a basic 
requirement of maintaining employment. 

The challenges from rapid technological change in the workplace are 
magnified by two major changes in the demographic composition of the U. S. 
workforce. First, declining birth rates in the United States are producing smaller 
cohorts of new labor force entrants resulting in increased demands on workers 
already in the labor force. Recent estimates suggest that about 75 percent of the 
workforce in 2000 AJD. [U. S. Department of Labor, 1987] will be made up of 
workers who are in the current workforce. Second, the sharp decline in new labor 
force entrants will be accompanied by larger concentrations of minorities and 
economically disadvantaged workers who enter the labor market with inadequate 
basic education and vocational training. Such demographic changes are likely to 
add pressure on businesses and government to develop new institutional 
mechanisms that provide basic and vocational skill training to adult workers 
throughout their working lives. 

Proponents of direct employer subsidies for employee retraining maintain 
that this type of government intervention is an effective means to encourage 
companies to respond to the challenge of technological and market changes. 
Although corporate expenditures on employee retraining are increasing in the U. S. 
and some leading companies are instituting new retraining policies, most companies 
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continue to treat employee retraining as a low priority in strategies to improve 
productivity and competitiveness [Sonnenfeld and Ingols, 1986]. This orientation is 
reinforced by the perceived risks and uncertainties of employee retraining and the 
absence of previous retraining experiences as well as by the absence of clear 
accounting and evaluation systems for estimating costs and returns on investment. 

In addition, companies often are slow to change their orientation to 
employee retraining. Organization researchers have shown that companies change 
their employment and training systems usually in response to some type of external 
pressure, such as technological change, changing product markets, reduced labor 
supply or sharply rising wage rates or other input costs [Osterman, 1987]. 
Companies can adapt to these pressures through a variety of strategies that establish 
different priorities to employee retraining [National Research Council, 1986]. 

Proponents of workplace-based retraining programs argue that government 
should serve a leadership role in encouraging companies to respond to technology 
and market pressures by retraining employees. These programs also should 
encourage companies to make long-term resource commitments to retraining by 
providing them with the risk capital to demonstrate the returns of employee 
retraining on company performance. 

Policy Issues. 

The job retraining issues set forth in this section give rise to the following 
policy questions pertaining to state-financed, workplace-based retraining programs: 

1) Should government financing of workplace-based employee retraining be 
limited to transferable skills that are expected to be in strong demand in 
state and local labor markets? 

One major policy rationale for government involvement in 
employee retraining is that employers will not undertake retraining in 
marketable skills 3 because their competitors may receive a free ride 
and because the company paying for the retraining will not achieve an 
adequate return on this type of training investment Government 
financing assistance is justified as a means for compensating the risk 
incurred in providing this type of training. 
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"Marketable skills" arc skills that may be transferred to and arc demanded by other employers in the worker's "effective" 
labor market, defined by the normal mobility of workers within a given occupation and industry. 
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This policy rationale is based on assumptions about 
externalities in employer-financed retraining that have been 
questioned recently by labor market researchers. In particular, some 
researchers have suggested that employers can receive a large share of 
the economic benefits from general skill training wbe? it is integrated 
with firm-specific training [Glick and Feuer, 1984]. In addition, the 
distinction between general and firm specific training and the 
determination of future demand for these skills in the labor market 
likely will be very difficult for government programs to establish in 
administering retraining projects. The problems of developing 
administrative guidelines for screening retraining proposals on the 
criterion of marketable skills will be addressed directly by this project 

2) Should government give priority to employed workers over unemployed job 

applicants with respect to training for new jobs within companies? 

The policy rationale for retraining employed workers is that 
such efforts will reduce the social costs of technological and economic 
change by reducing unemployment and unemployment insurance 
costs. An alternative strategy for government would be to provide job 
search and retraining assistance to the laid off workers and to assist 
the company in identifying qualified job applicants from outside the 
company. The first strategy, which is used in workplace-based 
retraining programs, is based on the assumption that retraining will 
prevent the unemployment of employed workers and will not have a 
major negative impact on the length of unemployment for qualified 
job applicants from outside the company. It also is based on the 
assumption that state programs can best minimize aggregate 
unemployment by preventing the job loss of experienced workers 
already employed by the company. However, these assumptions 
remain untested and are open to considerable debate. 

3) Are workplace-based retraining programs the most cost-effect z means for 

organizing and delivering job training to employed workers? 

The policy rationale for state-financed, workplace-based 
retraining programs argues, in part, that employers provide the most 
cost-effective mechanism for providing basic and vocational skills 
training to experienced workers. Workplace-based retraining 
provides: a clear connection between general skill training and 
specific job requirements; provides an alternative to workers who 
were unsuccessful in secondary or post-secondary schools; provides 
cost-savings on equipment and other training resources; and increases 
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market pressures on public training providers to become more 
responsive to business training needs. 

These policy assumptions on cost-effective training delivery 
have not been addressed in previous research and require further 
clarification before being addressed in this evaluation project. One 
major barrier to assessing the relative costs and benefits of workplace- 
based retraining versus institutional training is the absence of a 
common cost-accounting framework for training. This project will 
explore the problems comparing costs and training content and will 
clarify the major issues for further evaluation research. 

4) Should state-financed, workplace-based retraining programs give priority to 
non-professional, non-managerial workers? 

The policy rationale for state-financed, workplace-based 
retraining programs in part justifies government intervention on the 
need to provide ncn-professional, non-managerial workers with 
retraining opportunities. Government programs should favor those 
occupations that usually do not receive privately-financed education 
and training assistance in order to achieve a more equitable 
distribution of government and private training investments. By 
giving priority to these groups, government programs also reduce the 
likelihood that government funds will be substituted for business 
funds in employee retraining. If businesses generally do not spend 
money on training for certain groups of blue-collar and clerical 
workers, government programs that give priority to these groups will 
run a lower risk of substitution. 

Even in light of this policy rationale, there may be other 
reasons for government programs to provide retraining assistance to 
managerial and professional employees. These reasons will be 
explored in the evaluation project 
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HL ECONOMIC DEVELOPMENT POLICY ISSUES 



Whereas the states have a long history in education and employment 
training, they also have a long tradition of deliberately intervening in their 
respective economies in order to achieve greater community and individual wealth. 4 
However, states recently have expanded their economic development activities in 
conjunction with the adoption of a more aggressive definition of their economic 
roles. 

Fosler suggests that the emerging state - economic role is substantially 
different from the more conventional one in three important ways: 

In the conventional role, economic development is 
viewed as a government function (similar to police or 
health) whose principal mission is to recruit industry to the 
state. In the new role, economic development is conceived 
as a process that occurs predominantly in the market 
driven private sector, but is affected in all its phases 
creation, expansion, relocation, contraction, and 
regeneration — by a wide range of state actions, which cut 
across traditional functional lines. 

In the conventional role, the state passively accepts 
prevailing economic forces ( at most attempting to 
influence business location decisions), on the implicit 
assumption that national economic growth is more or less 
inevitable and, in any case, could be influenced only by 
federal policy. In the new role, the state employs an active 
strategy to improve its competitiveness by confronting and 
taking advantage of prevailing economic forces. 

In the conventional role, institutional responsibility 
for economic policy is consigned to a line agency of the 
state government, typically a department of economic 
development or commerce, whose principal mission is to 
recruit industry. In the new role, a fundamentally different 
set of institutional arrangements involving numerous 
organizations in both the public and private sectors and at 
various levels of government - is used to accommodate 



Louis H. Masotti, Professor of Management and Urban Development, Northwestern University, argues that government 
economic development programs are deliberate interventions in the affairs of the private sector with the ultimate purpose of 
achieving greater community and individual wealth than would be accomplished otherwise within a given time period. 
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the new strategic orientation, institutions that are more 
versatile and flexible in permitting the state to anticipate, 
specialize, experiment, integrate, evaluate, and adjust in 
dealing with new and changing economic forces. [Fosler, 
1988] 

In addition to changes in the role of. states with respect to economic 
development, states have broadened tf e scope of their efforts by making fuller use of 
three generic economic development strategies: business attraction, business 
enhancement and business engenderment. 5 Job training and retraining have become 
important components of both attraction and enhancement efforts. Essentially, 
states attempt to meet the labor demands of businesses by providing them with a 
sufficient supply of qualified workers. In taking this approach, human resources are 
regarded in much the same way as other production input resources that affect the 
qualities and prices of the final products. 
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Job Training in Busi ness Enhancement Prog rams 

The use of retraining in economic development is a relatively recent tactic 
becoming popular during in the 1940s [Stevens, 1986]. Nevertheless, its function ha^ 
undergone impdrtant changes over the last 40 years with the most recent significant 
shifts being staged by agencies such as Illinois' Prairie State 2000 Authority and 
California's Employment Training Panel. 

Since the 1960s [Stevens, 1986], the economic development use of job 
training has been primarily in the area of business attraction. In such 
circumstances, the state uses training to attract a firm, to prevent its relocation to 
another jurisdiction or to stimulate the expansion of a resident business. Other 
economic development tactics include infrastructure assistance, low interest loans, 
environmental waivers and special tax concessions. The precise mix of ' attraction 
incentives depends on their cost, the comparative advantages that they confer on the 
state over competing jurisdictions and their political palatability. In the aggregate, 
competition of this sort among states achieves nothing more than zero-sum results! 
States will copy each other's programs in order to avoid being placed at a' 
competitive disadvantage. Nevertheless, they have expended enormous amounts of 
resources in an attempt to eke out some slight advantage. 

5 Business attraction is the use of tote-financed incentives to attract a firm, to prevent its relocation to another junction or 
to stimulate the expansion of a resident businesses. Business enhancement is the use of state-financed assistance to enhance 
the competitiveness of businesses existing in the state. Business engenderment is use of state resources to encourage and in 
support of new business start-ups. 
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In one important way, the true role of job training assistance in attraction 
efforts has been clouded somewhat by the rhetoric of the programs. Frequently, 
program administrators describe their efforts in terms of the goals advocated by 
employment and training proponents: i.e., a reduction in unemployment and 
retention of jobs by making improvements in the skills of workers. The implication 
is that the individual workers participating in the programs are the intended primary 
beneficiaries. However, inasmuch as the use of these programs is to influence the 
decision of firms, the true beneficiaries are the businesses receiving the assistance. 

Although the beneficiaries of the programs often are obscured by the 
language of enabling legislation, job training programs used in attraction efforts 
often are perceived as achieving unambiguous and politically popular results - 
although many argue that these results are often specious. This is due, in part, to 
the fact that the very nature of the agreement between a state and the business 
being courted is performance-based: the state will provide specific in-kind or cash 
subsidies, for which, in return, the company agrees to relocate, expand or remain in 
the state. Although the company may have picked the state without being offered or 
provided such subsidies, the apparent association between the subsidy and the 
location decision suggests that the subsidies positively influence the company's 
decision. In the eyes of the public, the observable fact that a factory opens up or 
that more workers are going through a plant gate is sufficient proof that something 
was gained from the government subsidy to business. In addition, recent events 
suggest that in accepting government funds, the company also accepts certain 
obligations to the community and to the state. For instance, some companies have 
found that an attempt to leave a state after accepting its assistance may be viewed 
by the public and their elected officials as a breach of the terms and conditions of 
the subsidy. 6 

The job retraining programs exemplified by California's Employment 
Training Program and Illinois' Employer Training Assistance Program are 
representative of state economic development efforts aimed at enhancing the 
competitiveness of businesses existing in the state. Generally, these efforts may 
include broad adjustments to state tax or regulatory policies designed to improve the 
business environment and encourage investment. At the level of the firm, 
enhancement programs may include state preferential buying practices, technology 
transfer assistance, marketing assistance, special financing arrangements, special tax 
exemptions or relief efforts and targeted regulatory reform. 
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6 Eg,, Kenosha and Wisconsin state officials threatened lawsuits against Chrysler after the company sought to close the 
Kenosha facility. Public officials argued that in accepting state assistance, American Motors, the previous owner and operator 
of the plant, pledged itself to remaining in the state. In their view, Chrysler was obligated to keeping this pledge when it 
purchased American Motors. 
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Although more narrow in their scope than some enhancement-type programs 
(e.g., tax relief or limitations on product liability), programs like those operating in 
California and Illinois qualify for three reasons as important additions to this milieu. 

First, the retraining received by workers is essentially an investment intended 
to help the business achieve an improvement in its capacity to 
compete. Although only the human resources side of company 
operations is targeted by these new programs, the retraining is linked 
to other operational or technological changes. These changes are, in 
turn, designed to achieve a quality or price advantage. For instance, 
companies may adopt quality assurance systems (SPC), material or 
human resources management programs (MR?, qudity circles) or 
just-in-time inventory practices. 

Unlike the performance-based job training grant typical of the 
attraction efforts, the state is not seeking an explicit quid pro quo that 
the company will remain. Instead, the objective of the state is it 
wishes to achieve a stronger, more competitive business that functions 
above industry benchmarks based on key performance indicators. 
Examples of these indicators are such as unit costs, scrap costs, time 
of goods in production and individual worker productivity. Having 
met these standards, the company is presumed to be more likely to 
contribute to increases in community and individual wealth - the 
primary goals of economic development programs. 

Second, there is an underlying expectation at the operating agencies that 
intervention will establish a continuing practice at participating 
companies of integrating human resources development with their 
capital expenditures. This also is further evidence of the view that the 
primary result to be achieved from the retraining investment is an 
improvement in the relative position of the recipient firm. Such a 
view is based on a dynamic vision of the business environment and the 
actions that must be taken in order to survive; e.g., concurrent 
improvements in the skills of its workers as well as in production 
management systems or technologies. 

In light of limitations on state resources and the potential 
'demand for retraining assistance, it would be rational for the state to 
expect that the improvements achieved from the initial state 
retraining investment will produce sufficient resources within the 
company to pay for succeeding retraining efforts. In fact, the Prairie 
State 2000 Authority has imposed a one-time limit in its awards to 
firms (although this policy was loosened recently to permit companies 
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that once have received funds to seek assistance for retraining that is 
substantially different from the services funded initially). 



Third, by linking retraining subsidies to specific business purposes, programs 
such as those operated by California and Illinois begin to integrate the 
goals underlying employment and training policies with the goals of 
economic development. As was noted earlier, the objective of 
economic development is to help increase community and individual 
wealth. In earlier, attraction-type uses of job training, the state 
essentially is indifferent about the progress of the workers within the 
subsidized firm once the business makes good on its commitment to 
stay or expand in the state or to relocate to the state. However, the 
state financed, workplace-based retraining programs operated by the 
Prairie State 2000 Authority or the Employment Training Panel go 
much further in that each agency seeks assurance that the workers will 
acquire skills that will be put to productive use. 

In addition, it appears that these programs have tended to fund 
retraining for skills that will be applied directly by the recipient 
business as well by other businesses. For instance, both California 
and Illinois report funding many retraining projects in statistical 
process control (SPC) or materials resources planning (MRP). 
Although each project is tailored specifically to the company making 
use of the retraining, the basic principles learned by the workers in 
SPC or MRP may be transferred to a broad variety of manufacturing 
settings. In making the worker a more productive, flexible, and 
therefore, more valued employee, it would seem to follow that the risk 
of suffering prolonged displacement is somewhat reduced. Of course, 
such a result would match the primaiy goal of employment and 
training policies. 

/ilthough the use of job retraining as an enhancement strategy may be the 
basis foi linking economic development with employment and training, several 
thorny issues emerge. One issue is investment impact. Not only is the competitive 
standing of the firm affected presumedly by the several changes undertaken at the 
company, but it is affected also by changes occurring among its competitors [Porter, 
1985]. In addition, the timing of the effects from retraining and changes in the 
production management systems or technologies may vary by firm. Unlike 
attraction -type training subsidies that are tied to specific company performance 
within a given period of time, the timing of the returns from enhancement-type 
retraining investments cannot easily be pre-ordained. 
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Targeting and Substitution 



The argument in favor of state grant-in-aid programs to business is that such 
aid may be applied directly to companies where it will have a substantial impact 
without substituting government resources for private ones. Critics contend that this 
presumption is fundamentally flawed for three reasons: 

First, public policy makers and government officials possess neither accurate 
and timely information nor a reasonable basis for making correct 
decisions as to which industries should receive government assistance; 

Second, even if information on industries and businesses were available aud a 
proper decision model existed, many skeptics of targeted state 
retraining grants would argue that the process of making grant awards 
cannot be insulated sufficiently from outside political pressures. 

Third, government knowledge about the condition of the applicant firms is 
imperfect, creating an opportunity for public funds to be used in 
situations where the businesses are able and, but for the availability of 
government subsidies, willing to pay for the retraining themselves. 

Three generic strategies are examples of the range of economic development 
policies pursued by states in targeting government assistance by industry. One 
strategy promotes enhancement of key basic industries operating within the state. 
In this case, a "basic industry" is one that occupies a place that is relatively early hi 
the value-adding process and is both a major customer as well as vendor within the 
state's business community. 

Another strategy targets emerging industries that have no clear historic ties 
to the economic base. Such a diversification strategy has gained considerable 
popularity as demonstrated by the rush of states pursuing high technology industries. 

Finally, a state may either be indifferent on the issue of basic versus 
emerging industries or choose one over the other, but target as well specific 
businesses that export their goods and services out of the state or that produce 
goods and services that substitute for imported products. Under such a scheme, the 
state could rationalize such use of its resources in terms of the additional wealth 
that accrues to both the community and individuals. Similarly, there are a myriad of 
other strategies that use components of one or all of these three or that are entirely 
original. The aggregation of all development strategies describe the universe of 
such state efforts. 



ERIC 



34 

36 



Similarly! states also may invent various organizational structures to 
implement retraining programs* One factor affecting the organizational structure of 
a retraining program is the judgement by public officials as to the extent to which 
such a program must be insulated from outside political pressures. In California 
and Illinois, for instance, the two agencies offering assistance for workplace-based 
retraining are governed by independent, appointed boards. Other states may choose 
to operate such programs out of line departments. Irrespective of the solution, 
business, workers, the public and the press must be satisfied as to the legitimacy, 
efficiency and effectiveness of the grant awards process if the program ever is to 
become an effective economic development tool. Failing that, such programs end 
up as examples of failed partnerships between business and government 

However, the final criticism of discretionary grant programs pertains to the 
issue of government funds being used in place of private resources This criticism 
differs somewhat from the other two in that it calls bto question the capacity of the 
state agency to determine whether the company would undertake the appropriate 
retraining in a timely fashion without the assistance of the state. These questions of 
targeting and substitution are effectively two sides of the same coin: the efficient 
application of public resources. 

There are two necessary conditions to effective targeting by an agency 
offering government funds for workplace-based retraining.: 

First, the agency should be assured that retraining will have a positive effect 
on the company. Specifically, it should determine whether labor skills 
cited in the company's training plan are likely to be important 
components of the value-adding activities of the business. In deciding 
whether state-subsidized retraining will make a material difference in 
the competitive prospects of the company, it is important to determine 
affirmatively that such retraining would introduce productive skills 
into the work environment. 

Second, in targeting its resources government should assess the degree to 
which retraining funds are likely to substitute for private resources 
within the context of a targeted business. To some, the question of 
substitution hinges on the sources of government funds. Programs 
financed through a tax paid by employers in behalf of their employees, 
such like the one operated by California's Employment Training 
Panel, may be less susceptible to substitution questions than similar 
programs supported by general revenues. 

Inasmuch as funds for an employer tax-supported program 
come from a specific class of taxpayers who, in turn, are eligible to 
receive assistance, policy makers may regard the revenues for a state- 
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financed, workplace-based retraining program as coming from a 
"special services tax." Put in this context, a program funded by such a 
tax may not be regarded as necessarily competing with other programs 
for scarce general revenue. Companies seeking financial assistance 
from the program need only meet the program criteria established by 
the state. The result is that a tax imposed on the intended recipient of 
retraining funds tends to minimize the importance of the substitution 
question. 

However, there is no clear consensus that the true incidence of 
a unemployment insurance-type tax on employers is, in fact, the 
businesses that pay it. For instance, most labor economists would 
contend that the tax is paid by workers as a result of receiving lower 
wages. Such an argument may undermine the concept of the 
employer tax as a special services tax. 

Irrespective of the source of funds, whenever substitution becomes a material 
issue, the resolution of the issue seems to hinge on whether it is reasonable for the 
applicant business alone to bear the cost of retraining its workers. An assessment of 
the likelihood of business to finance worker retraining without government 
assistance seems to revolve around several concerns: 

1) the financial capacity of the firm to undertake the financing of the 

retraining; 

2) the risk that the firm will suffer from a free-rider problem; 

3) the risk that the retraining, in combination with specific technological 

improvements or organizational changes, will achieve anticipated 
changes in productivity and competitiveness; 

4) the likelihood that the skills to be learned by participating workers will be 

in demand within their effective labor market in the event that the 
current employer decides to lay them off; and 

5) the likelihood that businesses at the cutting edge of process technologies 

or management structures will find appropriately skilled workers 
within the existing labor pool. 

The likelihood of substitution will vary with the kind of industry and the 
underlying characteristics of the workforce in that industry. For instance, it is logical 
that the likelihood of substitution would be very low in very competitive industries 
requiring highly skilled, highly valued workers in emerging technologies, e.g., 
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machine tools. In order for the likelihood of substitution to be high, public or 
private educational institutions would have to take the highly unusual step of 
retraining workers in anticipation of the adoption of cutting edge technologies or 
management structures by area businesses within a given industry. In addition, the 
high demand for these skills increases the risk of member businesses to "free rider" 
problems. This may be aggravated further by high levels of competition within a 
given industry that may put the businesses within that industry in precarious 
financial circumstances. 

At the other end of the continuum are mature industries using widely 
available technologies operated by an ample supply of skilled workers. In such 
instances, retraining subsidies in these industries are not likely to result in new 
competitive advantages for resident businesses against foreign competition. 

Substitution considerations also extend to the firm level within given 
industries. The same criteria that are useful in assessing the likely level of 
substitution at the industrial level will apply to the firm level. Finns at the "low 
substitution" end of the continuum are those that demonstrate that there are real 
internal roadblocks to the timely retraining of their workers in a manner that 
achieves their business objectives. They also can show that there is a shortage of 
appropriately skilled workers and that the process technologies that these workers 
will encounter are sufficiently new so as to be beyond the scope of programs offered 
through the public education system that also is subsidized. Because of the newness 
of the technology, the firm also may be reticent to break new ground by becoming 
the first to retrain its workers in such systems. A firm at the "high substitution" end 
may simply find it more efficient to obtain the necessary skilled labor from the open 
market or from public educational and training institutions without having to incur 
any additional training costs. 



Policy Issuer 

Clearly, if state-funded, workplace-based retraining programs are to become 
a useful part of a state's package of economic development programs, they must be 
based on a better understanding of: 

(1) the conditions under which employee retraining is an effective 

intervention to improve company performance and worker 
productivity; 

(2) the role of employee retraining within a comprehensive business 

development strategy; and 
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(3) the conditions where the risk of substitution are minimized. 

Employee retraining is not the answer to all business development problems 
It seemr that it is most effective when important preconditions are met, such as a 
comprehensive business strategy and the presence of appropriate technologies and 
work practices. Consequently, in order for state-financed, workplace-based 
employee retraining programs to be effective, they should be based on clear training 
objectives that are tied to * comprehensive appraisal of the operations of each firm, 
particularly m light of their effect on overall productivity and competitiveness. Such 
* . mis also should be designed to maximize economic impact by targeting them 
to o^inesses that cannot accomplish their objectives without the timely assistance of 
th« state. 

The following policy issues are derived essentially from the economic 
development rationale for state-financed, workplace-based retraining: 

1) Should a government retraining program be based on the assumption that 
retraining is associated with improvements in the competitive 
performance and position of the participating business? The linkage 
between workplace-based retraining and a positive change in the 
performance and position of the participating business is open to 
question and should be established in. order for state participation in 
such retraining to be justified as an element of an economic 
development strategy. In light of the state's interest in achieving 
greater community and individual wealth, its intervention with 
retraining assistance should be considered successful if the recipient 
company has achieved an associated improvement in its competitive 
performance. 

Changes in competitive performance and position may be 
determined by comparing various individual and group attributes with 
industry benchmarks or standards prior to the commencement of 
retraining and at some point after the completion of retraining. The 
timing of the post-training evaluation would vary most likely with the 
nature of the skills being introduced into the workplace and with the 
timing of other events. 

In addition to developing appropriate assessment criteria, such 
pre- and post- intervention assessments wouL require states to adopt 
new data collection and analysis methods. Therefore, key objectives 
in the development of a comprehensive evaluation system are >o 
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define appropriate performance assessment criteria and to test the 
feasibility of states applying business performance assessments. 

2) Should the state require that the work environment at a company be altered 

in order to take advantage of state-financed skills retraining received by 
workers with state assistance? The underlying policy is that the state 
should maximize the long term benefits of its investment by insisting 
that the business make structural improvements in the work 
environment. 

This is based on the view that "productiveness" is not 
something that is carried solely in the heads and hands of workers. 
Other conditions generally must be present for workers' skills to be 
used efficiently and effectively. These conditions include the presence 
of proper tools and machinery in good working order and rational 
work rules that contribute to the success of the company as well as 
protect the health, safety and jobs of its workers. 

An assessment system incorporating this view requires that 
government, worker and company representatives are able to 
recognize the environmental conditions that are sufficient and 
necessary to achieving a productive workforce. A key objective in the 
development of an evaluation system is to establish a means for 
recognizing and a c sessing environmental conditions affecting 
productivity. 

3) To what extent should state-supported, workplace-based retraining be 

targeted to transferable skills demanded by the effe^. -2 labor market? 
To what extent should workers be pmvided with a thorough 
understanding of the underlying concepts for maximum flexibility? 
Should state subsidies emphasize workplace-based retraining where a 
likely outcome is an improvement in worker mobility? 

Occasionally, the state's interest in helping a particular 
company may conflict with broader public policy objectives. For 
instance, an employer often is dissuaded from making a retraining 
investment because it believes that the wages of its workers will be 
"bid up" by other businesses competing for their services or that the 
worker will leave for a better job. The state may encourage a business 
to implement technological and organizational improvements and 
make an investment in retraining by reducing the company's financial 
exposure by means of retraining subsidies. 
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However, the state also has an interest in encouraging other 
businesses in the same industry to adopt similar technological and 
organizational improvements by assuring them that there will be a 
sufficient supply of qualified workers to meet employer demands. In 
effect, the state wishes to encourage competition among businesses 
that it funds within its jurisdiction. This often will not work to the 
benefit of companies that initially deceive help. 

The problems faced by evaluators of state-financed, workplace- 
based retraining programs are that: 1) retraining assistance given to 
businesses adopting cutting edge technologies or practices lead 
demand for workers having particular kinds of skills, thereby denying 
the state an easy means of assessing transferability and labor market 
demand; and 2) the objective of mobility is possibly contradictory to 
performance-based contracts that base retraining assistance on 
worker retention. A skilled worker remaining on the job is expected 
to help the company achieve an improvement in its competitiveness. 
A comprehensive evaluation system should assess the feasibility of 
deterniining market demand and should accommodate for possible 
contradictions between employment and training and economic 
development policies. 

4) What types of businesses should be targeted for retraining investments? 

From an economic development perspective, the greatest net 
increase in wealth is achieved with government investments to those 
industries or businesses: a) that export much of their goods or services 
out of state; or b) that sell their goods and services to companies that 
use them as components of exported products; or c) that produce 
goods or services that will substitute for products imported into the 
state where there is competitive advantage with domestic products. 
Given this economic development imperative, states generally will 
seek to allocate scarce resources to companies that contribute to 
exports from the state or provide products that substitute for imports. 

States also provide assistance to businesses in order to achieve 
other economic objectives, such as the retention of a key employer 
within a community, the prevention of a plant shutdown or a 
reaffirmation of the faith of public officials in the economy. The net 
effect on the state economy from such retraining subsidies may be 
zero: while the state may enhance the condition of one firm, it may do 
so at the expense of other businesses in the state. 
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As an alternative, states already attempt not only to improve 
the overall condition of their economies by mitigating the negative 
effects of a shutdown of a major business on a given community by 
subsidizing the operations of the firm. In effect, such a scheme avoids 
social costs and is an acknowledgment of the immobility of certain 
worker populations and seeks to avoid prolonged employment 
dislocations by attempting to prevent them in the first place. 

Ultimately, these targeting questions are resolved individually 
by the states themselves. How they are resolved depends on the 
source of funds and the importance that the state places on giving its 
subsidies to businesses that will directly add to the collective wealth of 
their residents. Also, the question as to whether the state should 
chose between basic industries or emerging industries is a matter of 
policy and strategy for the individual state. In light of the variety of 
targeting objectives that may be used by a state, an evaluation system 
should be sufficiently flexible to provide state officials with an 
assessment of the performance of the program where such an 
assessment conforms with the state's own objectives. 

5) To what extent is the substitution of government for private funds a 
material issue? 

In some instances a prima facie case may be made that an 
investment of government retraining funds made in certain companies 
will produce a comparable net increase in the amount of retraining 
that occurs at the company. One set of examples is composed of 
companies that integrate retraining into overall business plans for 
improving their competitive positions and that clearly demonstrate 
financial need; e.g., businesses that are not paying dividends, that are 
not paying their executives excessive salaries nor are providing them 
with perquisites, or that are reinvesting all profits into their 
operations. Companies paying extraordinary dividends and executive 
salaries presumedly could afford to do the retraining on their own. 

However, there may be other real barriers to retraining that 
the state may be effective in dismantling. Between these two 
extremes is a point where government assistance is instrumental in 
prompting a business to provide its workers with the necessary 
retraining in a timely manner. This point likely varies by company. It 
is a function of many variables, including available cash on hand, 
investor expectations, previous employer experience with retraining, 
the likelihood that retraining would be successful, the risk that other 
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companies will hire away newly retrained workers, historical industry 
trends in worker retraining and the available supply of skilled labor at 
the present location and at other possible sites. In order to be assured 
that the state has not paid for straining that otherwise would occur, 
the aggregate effect of such variables should lead a prudent individual 
to conclude that retraining could not reasonably be accomplished in 
an appropriate and timely manner without outside subsidy. 

The problem posed for program operators and program 
evaluators is that a rule of prudence is derived from case law and not 
from a set of a prion standards. Therefore, it is important to test the 
feasibility of establishing a rule of prudence that is derived from case 
law. Assuming that such a rule is feasible, it then will be the task of 
developers of an evaluation system to determine the feasibility of 
assessing the extent of substitution, if any. 
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IV. POLICY ISSUES FOR EVALUATION 



As presented in Part A of this paper, state-financed, workplace-based 
retraining programs have arisen in the context of job training programs for displaced 
workers and economic development programs for business retention and 
development. Government essentially has two sets of interests that, depending on 
the policy objectives, coincide more closely with those of either businesses or 
workers. From the standpoint of economic development policy, the objective is to 
retain jobs in the state economy through improvements in the competitive standing 
of companies that operate in and from the state. However, from the standpoint of 
employment and training policy, government is more interested in retraining 
workers in skills that will improve their employment and earnings opportunities. 

In pursuing these twin objectives, the agencies operating state-financed 
workplace-based retraining programs have the difficult task of balancing the 
interests of businesses, workers and government ... all who share the costs and 
benefits of such programs. Businesses want workers with skills that enable them to 
work productively. Therefore, these workers are required to have a particular 
combination or package of skills that complement the specific technological or 
organizational environment of the workplace. Optimally, each business wants 
retraining to be customized to its special needs. Consequently, it will attempt to 
concentrate its expenditures on retraining activities that are expected to have direct 
effects on business performance. 

In contrast, workers are interested in learning skills that have a clear market 
value, both within the firm and generally within their effective labor markets. In 
learning these skills, these workers hope to maintain or enhance employment 
opportunities within the firm and improve their long-term employment and earnings 
prospects within these labor markets. 

In reviewing and extending the policy rationale and objectives of the state- 
financed, workplace-based programs operated by California and Illinois, this paper 
describes one view on the balance that these programs achieve among the 
competing interests of businesses and workers. It also describes how these programs 
are designed to reduce unemployment and retain jobs. Although this perspective is 
consistent with the general objectives and administrative guidelines of these state 
programs, it highlights some programs areas (e.g., certified competencies) that have 
received limited attention by the operating agencies. 

A 

In an effort to promote critical discussion, andestablish a clearand^bjective 
conceptual framework for evaluation, this section summarizes the-^i^Jvention 
model for state-financed, workplace-based retraining programs. This intervention 
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model describes the program outcomes that are necessary for state retraining 
programs to be successful in reducing unemployment and retaining jobs. This 
overview of the intervention model is followed by a brief discussion of targeting and 
substitution issues in making business funding decisions. 



Intervention Mn del 



Most state-financ«d, workplace-based retraining programs are designed to 
reduce unemployment and retain jobs by retraining employees who are at risk of 
losing their jobs because of changing skill requirements in the workplace. They are 
based on the assumption that employee retraining reduces the incidence of 
unemployment and retains jobs by improving the productivity of workers. These 
programs assume that worker productivity is a product of both worker skills and the 
integration of these skills into a productive workplace. A productive workplace, in 
turn, is a product of a variety of factors including process technologies, job design, 
management-labor cooperation, compensation systems, and employment security 
policies. To be successful in reducing unemployment and retaining jobs', worker 
retraining must be tied integrally to larger workplace changes that are intended to 
improve work unit or company performance. 

These programs are based on the rationale that the risk of unemployment is 
reduced and jobs are retained when workers are retrained in marketable skills that 
are successfully integrated into a larger company initiative to improve the 
competitiveness of the firm. On the one hand, an assumption underlying state- 
financed, workplace-based retraining programs is that jobs are retained when 
workers are retrained in conjunction with changes in process technology end work 
organization that are intended to improve the productivity of the work unit and the 
company relative to competitive standards in the industry. It is assumed that the 
state employment base of the company will remain stable if the company is 
successful in achieving or enhancing its competitive advantage through worker 
retraining. On the other hand, another underlying assumption is that the risk of 
future unemployment of retrained workers may be reduced as a function of workers 
achieving certified competencies in skills that: (1) contribute to improvements in 
work unit and company performance that, in turn, stabilize the employment base of 
the company, and (2) improve the likelihood that retrained workers will be able to 
find new jobs with these skill competencies if these retrained workers are dislocated 
because companies were not successful in maintaining or enhancing their 
competitive standing. 

The intervention model for state-financed, workplace-based retraining 
programs is illustrated in Figure 1. 
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Figure 1 

Intervention Mode! for Evaluation 
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The expected outcomes associated with workers achieving certified skill 
competencies in new process technologies or work processes are improvements in 
work unit performance. Improved work unit performance should produce 
corresponding improvements in company performance. In addition, worker 
retraining also is expected to maintain or increase worker earnings due to 
improvements in worker productivity and to an increased likelihood that retrained 
workers may be able to find comparable or better employment opportunities with 
other employers. 

A successful retraining project is expected to result in the following 
performance outcomes: 

o Certified Skill Competencies. Retrained workers achieve certified skill 
competencies that are designed to improve work unit performance 
and enhance *the employment opportunities of workers outside the 
company. 

o Work Unit Performance. Worker retraining is associated with 
improved performance of work units within the company. 
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o Company Performance. Worker retraining and associated 
Improvements in work unit performance are related to improvements 
in company performance. 

o Worker Earnings. Worker retraining results in stable or improved 
employment and earnings experiences for retrained workers. 

Targeting and Substitution Issues 

The effectiveness of the intervention model in reducing unemployment and 
retaining jobs - is dependent on the ability of state agencies to target government 
assistance to businesses where it will have significant direct impact without 
substituting government resources for private ones. In light of the policy rationale 
and intervention, model of state-financed, workplace-based retraining programs, 
three major targeting and substitution issues are important to the development of 
administrative guidelines for business funding decisions: 

■ 

Competitive Strategy and Company Performance. The intervention model 
states that worker retraining will reduce unemployment and retain 
jobs when it is used as part of a larger company strategy to improve 
company performance and competitive standing. Therefore, state 
agencies should consider giving priority to those companies in which 
training is a critical component in their competitive strategy and is 
integrated with other workplace changes (e.g., technology) related to 
performance. As discussed earlier, one major economic development 
policy issue is whether state programs can effectively screen 
businesses based on a statement of business strategies and 
performance objectives. 

Training in Marketable Skills. The intervention model al& ) states that worker 
retraining will reduce unemployment and retain jobs when workers 
receive transferable skills that are in strong demand in state and local 
labor markets. In addition, substitution risks will be minimized 
because businesses will face greater "free-rider" risks and 
uncertainties and will be less willing to undertake training without 
some type of government assistance. Based on this assumption, state 
agencies should target those businesses that are retraining workers in 
marketable skills, especially where there is a recognized or 
anticipated skill shortage in state labor markets. Whether state 
programs can effectively identify marketable skills and potential skill 
shortages in state labor markets remains an important policy issue. 
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Financial Capacity and Commitment. As discussed in the economic 
development policy section, the most critical substitution issue is 
whether.companies have the financial capacity and commitment to 
finance worker retraining without the assistance of government State 
agencies should consider giving priority to companies that provide 
some type of assurance that the company would not undertake worker 
retraining in the necessary timeframe without government assistance. 
How state agencies should evaluate company statements of financial 
need is open to considerable debate. 

The intervention model discussed in this section and these targeting and 
substitution issues provide the basis for developing a monitoring and evaluation 
system for state-financed, workplace-based retraining programs. 
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PART B: EVALUATION ISSUES 
I. PROGRAM MONITORING AND EVALUATION 



A permanent monitoring and evaluation system for state workplace-based 
retraining programs similar to those in Illinois and California should consist of three 
complementary and interrelated types of evaluation activities: (1) process 
monitoring; (2) performance assessment; and (2) impact evaluation. These three 
types of evaluation activities are complementary in the questions they address and in 
the functions they serve in the assessment of state workplace-based retraining 
programs. [Rossi and Freeman, 1985] 

o Process monitoring determines whether workplace-based retraining 
programs are operated in accordance with stated program objectives 
and intervention models that are assumed to produce the desired 
program effects. 

It is directed at two major questions: (1) whether retraining programs 
are reaching the appropriate target population of businesses and 
workers, anu (2) whether retraining programs are following 
administrative guidelines that minimize the possibility of substitution 
effects. 

o Performance assessment determines whether a program has achieved 
the specifi: outcomes that accomplish the goals and objectives of the 
program* It asks whether the program has achieved the gross 
outcomes tl:pt are necessary to produce the desired effects. 

o Impact ussessment drccks the validity of the intervention model; i.e., 
whether observed outcomes can be attributed to the intervention. T n 
the case of workplace-based retraining programs, one such assessmu 
may be to validate the cost-effectiveness of such programs in 
achitving a "net reduction 1 in unemployment or a "net increase" in job 
retention. 

Once the policy issues and intervention models have been anicu! ;ted, the 
first step in the development of a monitoring and evaluation system for state- 
financed, workplace -based retraining programs is to establish a process monitoring 
and performance assessment subsystem. Such a subsystem determines the extent 
lhat actual program operations: 1) conform to administrative guidelines on 
targeting and substitution; and 2) achieve the necessary intermediate and final 
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outcomes for the programs to have a significant impact on unemployment and job 
retention. 



After such a process monitoring and performance assessment subsystem is 
firmly in place, the next step is to consider the feasibility of a net impact assessment 
subsystem. This subsystem measures and estimates the effects of the program in 
reducing the incidence of unemployment and retaining more jobs in the state 
economy. 

Process monitoring and performance assessment are important preconditions 
for effective net impact evaluations. Although the policy issues forming the basis of 
an impact assessment generally are considered to be the most important, this type of 
evaluation is veiy difficult to design and implement This is difficult especially for 
innovative programs undergoing continual development and change due, in part, to 
continued improvements in the sophistication of program operators. In addition, 
these programs are difficult to manipulate so as to achieve the required research 
designs. 

Design Issues 

In developing appropriate and feasible administrative guidelines and 
performance outcome measures for state-financed, workplace-based retraining 
programs, four major design issues should be considered: 



o Policy Consistency. Consistency with the policy rationale, 
intervention model, and actual operation of state-financed, 
workplace-based retraining programs. 

o Program Application. Effectiveness of the administrative 
guidelines and program outcome measures in improving 
the management and evaluation of state programs from the 
standpoint of program administrators and major program 
sponsors and stakeholders. 

o Measurement Capacity. Capacity for valid and reliable 
measurement using existing or readily accessible data or 
data gathered through widely used testing and survey 
materials. 

o Consistency with Private Sector Practices. Sufficient consistency 
with exemplary private sector practices so as to not impose 



ERLC 



jo 51 



unreasonable costs or time delays and reduced interest in 
program participation. 

The first three design considerations raise ordinary problems and issues in 
the development of administrative guidelines and performance outcome measures 
for public programs. Minimally, the evaluation system should flow logically from 
the policy rationale and intervention model that form the basis of the program. 
These administrative guidelines and performance measures also should be 
pragmatic, easily understood and measurable. 

Inasmuch as retraining is conducted within firms, state agencies should 
ensure that these administrative guidelines and performance measures are 
consistent with private sector practices. Also, these programs depend on voluntary 
business participation and are designed to encourage and facilitate retraining 
initiatives in the private sector. Therefore, the reporting requirements from an 
evaluation system should be based in part on information that companies 
realistically should be expected to collect as part of their own evaluation and 
monitoring efforts. In the event that the state is not consistent with leading private 
sector evaluation practices, the result will include added costs to the participating 
businesses, time delays and reduced business interest. 
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EL PROPOSED ADMINISTRATIVE GUIDELINES FOR BUSINESS 
SCREENING 



A major policy problem for state-financed, workplace-based retraining 
programs is achieving an efficient and effective method for targeting training 
investments to businesses and workers where such investments will have the greatest 
impact As presented in the policy discussion and the discussion of the intervention 
model, retraining projects will have their greatest impact when: 

(1) worker retraining is integrated into larger company plans to improve 

company performance and maintain or enhance its competitive standing 
within its industry; 

(2) worker retraining is a critical factor in improving company performance 

because of significant changes in skill requirements; 

(3) retraining provides workers with transferable skills that are likely to be in 

high demand in their effective labor markets; and 

(4) companies are unlikely to undertake the retraining without outside 

intervention such as government assistance. 

Although retraining programs may establish a broad variety of administrative 
guidelines a*> the basis for a process monitoring system, the administrative guidelines 
for business screening - business selection and substitution minimization - should 
be applicable in most programs. 



Selection Guidelines 



State-financed, workplace-based retraining programs 
should establish clearly defined screening guidelines for 
evaluating company retraining proposals. These 
guidelines should address: (1) company performance 
objectives relative to industry performance standards; 
(2) documentation on existing skills of workers relative 
to new skill requirements; and (3) the projected 
demand for new skills in the effective state labor 
markets. 
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_ State retraining programs first should establish administrative guidelines for 
selecting only those companies that can meet the first three conditions for program 
impact As shown in Table 1, these conditions provide the basis for selection 
guidelines that can be used to assess whether it is appropriate for the company to 
receive retraining assistance. 

In addressing the first selection guideline, itate programs wouid assess a 
company retraining proposal on whether the company has designed worker 
retraining that would contribute to improvements in company performance, and 
maintain or enhance competitive standing. As presented in the discussion of policy 
issues, businesses would be expected to provide a brief project plan that explains the 
proposed worker retraining in the context of major changes in process technology or 
work organization and that sets forth clear and measurable objectives that are 
relevant to their competitive strategies. Each set of project plans would explain the 
importance of each performance indicator (e.g., unit costs, quality standards) in the 
context of the associated competitive strategy and in the performance benchmarks 
that are generally accepted by the industry. These company performance 
expectations would be established jointly by businesses and state retraining 
programs. 

The major feasibility issue for this selection guideline is the ability of the 
company to define measurable performance goals and their ability to relate these 
performance goals to objective industry performance standards. A related issue is 
the capacity of state program staff to utilize readily available industry information in 
evaluating these company retraining proposals. Although state business retention 
and development programs have become involved in assessing business strategies 
and competitive problems, only limited work has been done on techniques for 
defining and measuring competitive standing. Nevertheless, in developing 
competitive strategies, many companies typically gather information on their 
competitive position relative to other businesses. Although this information is often 
informal and may be based on a small sample of competitors, it is critical in setting 
reasonable and effective goals when planning major changes in technology and work 
organization. 

In light of the need to formalize and systematize this type of information 
within an industry, many countries (e.g., Canada and Japan) as well as some 
industry associations in the United States have collected and presented comparative 
data to companies to assist them in strategic planning. One.example of these efforts 
is the Interfirm Productivity Comparison (IFC) approach that has been tested by the 
machine tool and electronics industries in the United States [Bearse, 1987]. Using 
this approach, businesses within a particular industry submit productivity and 
performance information (e.g., labor productivity measures) to a government agency 
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or to a major trade association. These data are aggregated and analyzed with the 
result being a set of industry performance standards. In return, the company 
receives a report that compares it to the average performance levels of businesses 
within its product area and size class. This information allows companies to analyze 
overall company performance in terms of comparative information in detailed 
performance categories. 

This general approach demonstrates that a state program may stimulate state 
economic development agencies and industry trade associations to promote the 
accumulation and dissemination of comparative information for industries and 
businesses targeted for retraining investments. This example also suggests that 
comparative company information will improve planning and coordination by state 
employment and training and economic development programs and provide 
industry trad*, associations with a clear role in state economic development and 
employment and training efforts. 

Although the BFC approach provides a useful starting point, it has a number 
of limitations that must be explored in the project. First, the BFC productivity 
indicators in their current form are too broad to provide a useful comparison for 
specific company strategies. Therefore, these measures will need to be refined and 
made more flexible if they are to accommodate the project proposals typically 
received by state retraining programs. Second, the IFC framework, or a similar 
interfirm comparison method, may be too costly to implement and operate. If costs 
are too high then an alternative planning and evaluation tool for state retraining 
programs will need to be developed. 

The second selection guideline on changing skill requirements would require 
state programs to document such changes and may prompt them to determine 
whether the workers require retraining to perform their new jobs. It is uncertain 
whether companies" are able to or would be willing to define and measure existing 
skill levels of workers relative to changing workplace requirements. Indeed, it may 
be too difficult for some companies to document sufficiently the new skill 
requirements coming from changes in process technology or work organization. In 
order to answer these feasibility questions, companies will be abked about their 
willingness and ability to do skill testing. In addition, examples will be sought of 
companies in similar industries that have done skill testing and assessment in their 
own retraining programs. 

The third selection guideline would assess whether state-financed, 
workplace-based retraining program are retraining workers in marketable skills that 
are projected to be in demand in the workers' effective labor markets. This raises 
questions on the capacity of state retraining programs to project or certify skill 
shortages in a given labor market. There typically are three sources for this 
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information: (1) industry or trade association information on retraining needs, (2) 
state or local labor market information on occupational supply and Qc-iand, and (3) 
direct employer telephone surveys on training and hiring needs. All three sources of 
market demand information will be assessed as part of the case studies on the basis 
of ease of access, utility and cost. 

Although other possible targeting guidelines are suggested in Part A of this 
paper, none appear to have universal application. States will use a variety of other 
targeting criteria depending on their economic development policies. In light of this 
variety, such industry and firm targeting issues are put beyond the scope of this 
evaluation project and yill not be addressed directly in the case studies. 

State-financed, workplace-based retraining programs 
should establish planning and administrative 
procedures that will minimize substitution effects in 
retraining projects. These procedures should address: 
(1) company's statement of financial need or 
reluctance; (2) the degree of retraining in marketable 
skills; (3) previous retraining expenditures by the 
company; and (4) disinvestment decisions that would 
affect the state economy. 

In addition to establishing clearly defined screening guidelines for evaluating 
company retraining proposals, state-financed, workplace-based retraining programs 
also have the difficult and important task of establishing effective safeguards against 
substitution; i.e., the funding of retraining projects that would have been done at the 
same level and in She same timeframe through private funds even without 
government assistance. Limitations on the use and capacity of government 
resources, practical political considerations and the desirability of tying state 
expenditures to specific outcomes will require state agencies to establish planning 
and review guidelines for assessing the capacity of an applicant company to use its 
own resources in retraining workers. Minimally, state agencies likely will require 
written assurances from the company that government assistance is required to 
achieved the de 3d level of retraining in a timely fashion. State-financed, 
workplace-based retraining programs in California and Illinois already require such 
written assurances from participating companies. 

However, at least four additional types of documentation should be 
considered in evaluating company retraining proposals. In addition to a written 
assurance on the need for government assistance, companies also should provide 
accompanying documentation on at least one of the following: (1) financial need or 
general corporate reluctance to undertake such regaining;; (2) the absence of 
previous refraining expenditures in similar skill areas with similar workers; (3) 
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substantial shift in skill requirements requiring retraining in marketable skills; or 
(4) efforts to forestall possible company disinvestment decisions (See Table 1). 



The most difficult substitution guideline to address is the degree to which 
retraining projects are training workers in marketable skills that are in demand in 
the worker* s effective labor markets. The clearest indicator of skill marketability is 
evidence of demand for that skill outside the company (e.g., secondary labor market 
information, want ads, solicitation by other companies for comparably skilled 
workers). Hie policy section argued that companies would be less likely to conduct 
worker retraining without government intervention in situations where they were 
helping workers to acquire marketable skills that could be lost by the company to 
other employers. In this situation, such retraining programs approach the same 
level of substitution as that of retraining programs offered by government-operated 
providers (e.g., community colleges, vocational education institutions). 

The feasibility of this screening guideline will be assessed using three sources 
of market demand information: (1) industry or trade association information on 
retraining needs, (2) state and local labor market information on occupational 
supply and demand, and (3) direct employer telephone surveys on retraining needs. 

As shown in Table 1, the major feasibility questions that will be explored in 
the case studies address the capacity of companies and state retraining programs to 
develop and document the required information during the application process. 
These feasibility questions will be examined through reviews of retraining proposals, 
business site interviews, interviews with program staff and reviews of available state 
and local labor market and economic data. 
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m. PROPOSED PERFORMANCE INDICATORS AND RELATED 
RESEARCH QUESTIONS 



A performance assessment system is based on a set of outoame indicators 
that may be associated with each objective of a given program. The performance 
measures described below establish the program as the unit of evaluation and are 
derived by aggregating the performance outcomes of separate company retraining 
projects. These company retraining projects will provide the basis for the case 
studies in this evaluation project The major feasibility questions and research 
activities for the case studies are listed in Table 2. The specific questions addressed 
in each research activity are provided in the last section. 

Certified Competencies in Marketable Skills 

Based on the intervention model discussed in Part A, state-financed, 
workplace-based retraining programs may be evaluated partially on the degree to 
which workers end up with certified competencies in marketable skills that are 
critical in improving work unit and company performance and are in demand in the 
workers'' effective labor markets. Measures of transferability and market demand 
are discussed in the previous section as business screening issues. Performance 
outcome measures address the level of competencies in skills identified in the 
retraining plan as exhibited by workers at the end of the retraining project 

State-financed, workplace-based retraining programs 
should assure that workers are tested on marketable 
skill competencies before and after retraining with the 
program evaluated on the percentage of workers 
achieving the required skill competencies established in 
the retraining project plans. 

Competency-based training and evaluation has been proposed and used in 
employment and training programs for many years. It has been applied in 
secondary vocational education and Job Corps and is now being implemented as a 
basis for evaluating youth programs in JTPA. Competency-based training and 
evaluation also has been identified as a major area of business evaluation activity 
[Dvorin, 1986] especially in companies that have instituted knowledge-based 
compensation systems [Lawler and Ledford, 1985]. Because of this previous 
evaluation experience in both government and business, this program outcome 
measure should be given detailed consideration in the evaluation of state-financed, 
workplace-based retraining programs. 
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Work Unit Performance 



State-financed, workplace-based retraining programs may be evaluated on 
the degree to which worker retraining is associated with improvements in work unit 
performance consistent with the objectives identified in the retraining plan. 

State-financed, workplace-based retraining programs 
should assure that work unit performance is assessed 
after retraining. The program should be evaluated on 
the percentage of work units achieving the performance 
objectives established in the retraining plan. Work unit 
performance should be evaluated after 90 days of on- 
the-job experience. 

An objective in most employment and training programs is to achieve gains 
in the wages and earnings of workers through improvements in productivity. 
Although most employment and training evaluations do not measure worker 
productivity directly, Bishop has proposed that productivity improvement measures 
be included in program evaluation efforts for employment and training programs 
[Bishop, 1987]. Similar approaches have been suggested in training evaluation of 
business-operated programs [Dvorin, 1986]. This study will test the feasibility of 
using work unit performance measurement* as the basis for assessing productivity 
improvements. This test will be done in light of serious problems with supervisor- 
based performance appraisal and given the underlying assumption that productivity 
is largely a phenomenon of the work unit Such measures will be taken after the 
completion of retraining and will be assesse4 against the performance expectations 
established in the company retraining proposal. 

Some state-financed, workplace-based retraining programs, especially those 
operated in California and Illinois, already evaluate retraining projects based on the 
retention of retrained workers 90 days after retraining. The scheme proposed above 
expands the assessment to include an appraisal of work unit performance in light of 
pre-training expectations. 



Company Performance 

State-financed, workplace-based retraining programs may be evaluated in 
terms of the stiength of their association with improvements in the competitiveness 
of participating businesses. As discussed in Part A, businesses should be expected to 
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provide brief project plans that explain the roles that worker retraining will play in 
the context of major changes in technology or work organization. Such plans would 
state these roles in terms of improvements in company performances as marked 
.against ' clear and measurable objectives that are important to their current 
competitive standing within their respective industries. These company 
performance expectations should be established jointly by businesses and state 
retraining agencies and should be based on objective industry benchmarks that 
provide external validation for minimal industry standards. These screening issues 
were discussed in the previous section. 

State-financed, workplace-based retraining programs 
should assess improvements in company performance 
relative to objective industiy benchmarks 
approximately 1-2 years after completion of retraining. 
These programs should be evaluated by the percentage 
of companies that achieve their planned performance 
goals that were established relative to industiy 
benchmarks in the project plan. 

The evaluation of job retraining investments in terms of returns to company 
performance is gaining acceptance by businesses, especially by large corporations. 
There is a growing trend to apply organizational performance standards and results 
as the criteria for training evaluation [(Dvorin, 1986; Robinson, 1987; Nickols, 
1979)]. A major problem with this approach is that it is difficult to isolate the effects 
of retraining from other workplace changes that also contribute to changes in 
company performance. In a performance assessment system (as opposed to a net 
impact evaluation system), there are major concerns regarding the validity and 
reliability of outcome measures and with the logical relationship between certified 
worker competencies and work unit or company performance. 

Wages and Earnings of Workers 

State-financed, workplace-based retraining programs may be assessed on 
their effect on the wages and earnings of workers. Improvements in workers' skills 
and job performance, especially when they contribute to improved company 
performance, should result in a reduced incidence in unemployment and improved 
annual earnings. 

State-financed, workplace-based retraining programs 
should assess improvements in annual earnings among 
retrained workers approximately 1 year after 
completion of retraining* 
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Previous evaluations Of federal and state employment and training programs 
have assessed mainly the effects of job training on the wages and earnings of 
workers. This type of performance monitoring already has been implemented by 
state workplace-based retraining programs (e.g., Moore, Wilms and Bolus, 1988) 
using state Unemployment Insurance records] A number of researchers have 
questioned whether suchearnmgs.information provides clear, unbiased estimates of 
the effectiveness of government training programs [Bishop, 1988]. In addition, a 
number of conceptual problems are encountered when pre- and post-training 
earnings are used to assess dislocated worker programs and state workplace-based 
-retraining programs when they retrain experienced workers with extensive previous 
work experience and high-wage rates. In some cases, retrained workers will be 
placed in new jobs in which they have no previous work experience or skill training 
and which provide similar or lower wage rates. The major conceptual problem is 
the assumption that prior earnings are a valid indicator of the earnings that workers 
wou'd receive in the absence of retraining. In many cases, prior earnings provides 
an inflated estimate of expected earnings in the absence of retraining. Comparison 
groups of workers who did not complete retraining also provide biased estimates 
because of unmeasured differences between those workers who did not complete 
framing. These conceptual and methodological issues will be addressed and 
illustrated in the case studies. 




W. NET IMPACT EVALUATION AND RELATED RESEARCH 
ISSUES 



The history of net impact evaluation in employment and training programs 
over the last 10 years has been marked by a continuing debate on the relative 
strengths and weakness of experimental versus quasi-experimental research designs 
(e.g., Barnow, 1986; Burtless and Orr, 1986). Experimental designs require the 
random selection of program participants from a pool of eligible potential 
participants. In contrast, quasi-experimental designs develop matched control 
groups for comparison with program participants. While random assignment is an 
obvious advantage in establishing a research design with strong internal validity, 
classical experiments have three major disadvantages that make them inappropriate 
for most state workplace-based retraining programs: 

First, classical experiments are extremely difficult and costly to implement 
because they require extensive and continual field work to maintain 
the research design; 

Second, they tend to change the very nature of the programs they are 
designed to evaluate; 

Third, and most important, classical experiments require that program deny 
or significantly delay services to a random set of businesses or 
workers. This is extremely difficult to justify in ongoing programs, 
especially programs that continually confront sensitive decisions on 
which businesses should receive state retraining funds. 

Based on the major problems associated with classical experiments, quasi- 
experimental designs appear on their face to be more feasible as the basis for an 
impact evaluation system. 

The major advantage of net impact evaluation for prevention programs such 
as state-financed, workplace-based retraining programs is that program objectives 
can be measured directly. In the case of state workplace-based retraining programs, 
the twin objectives of job retention and reductions in unemployment may be 
evaluated through comparison group designs. In addition, two additional program 
effects that are difficult to assess through gross program outcomes - earnings gains 
and reductions in unemployment insurance expenditures - may be measured more 
accurately through comparison group designs. 
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Although comparison group designs have been used extensively in the 
evaluation of employment and training programs (e.g., Barnow, 1986; State of 
Washington, 1987), they have yet to be developed for programs that retrain 
employed workers. The major problems in achieving such an evaluation are in the 
development of comparison groups of businesses and workers and in the 
identification of major business and worker characteristics that must be controlled 
before an estimate of the impact of the program may be made. 

State-financed, workplace-based retraining programs 
should explore quasi-experimental designs in 
developing impact evaluation capabilities. States 
should consider four major impact measures: (1) net 
job retention in participating companies; (2) direct 
and indirect economic impact of retained jobs on the 
state economy; (3) net earnings gains of retrained 
workers; and (4) net savings on unemployment 
insurance. 

In exploring the feasibility of net impact assessments using quasi- 
experimental designs, this study will address the methodological and data problems 
that state programs would face in developing valid comparison groups of businesses 
and workers and how they could collect reliable comparison information on the 
recommended impact measures: 

(1) What are the major sources of business data that could potentially be 

used to build reliable comparison groups (e.g., company applicant 
files, state UI files, Dun and Bradstreet, industry association surveys 
such as IFC)? 

(2) What major business and worker characteristics should be measured in 

estimating net impacts through comparison group designs? 

(3) What are the major data limitations that should be addressed in 

measuring these business and worker variables? 

(4) What is the most appropriate comparison group design given these data 

limitations? 

(5) What are the potential administrative costs to state programs and 

pariicipating businesses in operating such an impact evaluation? 
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V. PROPOSED RESEARCH DESIGN FOR CASE STUDIES 



Based on the discussion of policy and evaluation issues, this project will 
conduct 24 case stucfies of retraining projects that have been funded by state- 
financed, workplace-based retraining programs. These oases will be selected from 
retraining projects funded by California's Employment Training Panel and Illinois' 
Prairie State 2000 Authority (ETAP). Additional cases will be selected from similar 
programs or projects in other states. These case studies will be designed to assess: 
(1) the accuracy of the intervention model presented in Part A of this paper in 
describing the actual operation of state-financed, workplace-based retraining 
programs, and (2) the feasibility of establishing the proposed administrative 
guic^.jLJies, program outcome measures, and net impact measures for a monitoring 
and evaluation system. This section describes the proposed sampling design and 
field work activities for the project case studies. 

Project Sampling Design and Research Activities 

A two-step sampling design for selecting retraining projects will be used to 
assess the accuracy of *he intervention model and the feasibility of the proposed 
mc litoring, performance, and evaluation indicators. The population of retraining 
projects that will be considered in the case studies is all retraining contracts that 
have been funded within the last two years and that are scheduled to be completed 
at least 90 days before the end of the case study period. In addition, retraining 
projects must meet the following conditions: 

o Retraining funds must be awarded to companies and these companies 
are given the authority to select the type of retraining and the training 
provider. 

o The retraining funded by the state programs must involve vocational 
skill retraining away from the normal routines of work. 

o The retraining project must address changing skill requirements based 
on technology change or job restrucUiring. 

o The purpose of the retraining project is tied clearly to improvements 
in company performance that can be measured by a performance 
indicator (e.g., unit costs, quality standards). 

The first step will include a brief review of all retraining contracts with 
companies that meet these conditions in California and Illinois. It also will include 
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a review of the policies and procedures of the state program and interviews with 
program staff as a means of understanding the application of these policies and 
procedures in the planning end completion of each retraining project. 

Based on the program review and these interviews, a purposive sample of 
state retraining projects will be drawn with the assistance of the agency staff. This 
sample will be composed iof projects that best illustrate the policy rationale and 
intervention model developed in this paper. It also will be selected to illustrate the 
application of administrative guidelines in solving business selection and 
substitution problems. Finally, the retraining projects that are selected will be the 
best demonstration of the potential of measuring the program outcomes and 
impacts proposed in previous sections. To the extent possible given any limitations 
on the variety and scope of possible case studies, the purposive sample will provide 
a representative array of projects in manufacturing and non-manufacturing firms as 
well as large (500 and over employees) and small (less than 500) firms. At least one 
case in the sample will be a project that was completed at least 1 year prior to the 
time of study. This will permit the measurement of program outcomes on trainee 
earnings and company performance. 

Phase I: Program Review and Exploratory Case Studies 

The project case studies will be conducted in two phases. The first phase will 
begin with a review of all retraining projects meeting the project sampling criteria 
discussed <toove. It also will include an exploratory exaiuination of three of the 
sampled cases in California and Illinois. The purpose of this exploratory work is to 
test the policy rationale and intervention model and to gain a rough assessment of 
the feasibility of the administrative guidelines and outcome and impact measures, 
proposed in this paper. A final set of administrative guidelines and outcome and 
impact measures for state-financed, workplace-based retraining programs will be 
developed based on the results of this exploratory phase. The revised evaluation 
system will be used on the remaining case studies, including the additional cases 
from other states. 



Exploratory/ Case Studies. 



The exploratory project case studies will consist of six separate steps for each state 
agency involved in the study: 

t 
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Retraining Project Review. As discussed above, the firs}: major work task will 
be u review of all state-financed, workplace-based retraining projects 
meeting the sampling criteria. Ihe purpose of this review is to collect 
secondary information on: (1) company performance goals; (2) skill 
retraining areas; and (3) type of training provider in order to assess 
the feasibility of defining and measuring company performance and 
general skill competencies and to assess the representativeness of the 
projects included in the detailed case studies. The program staff also 
will be interviewed on each retraining project in order to assess the 
applicability of the policy rationale and intervention model to each 
retraining project Based on this project review, a purposive sample 
of projects will be selected for detailed case studies. After selecting 
the training projects, the results achieved from the overall review will 
be used to re-assess the overall representativeness of t ie purposive 
sample and the limitations that such a sample will impc s on future 
conclusions* 

Company Background Review. The second step will involve background 
research on the companies selected for the case studies. Current 
business information on the company and major trends and shifts in 
the industry will be reviewed. Also included in the review is 
information provided by national and state trade associations. The 
purpose of this background review is to prepare for the business 
interview sessions and identify potential sources of comparative 
industry information for establishing industry benchmarks. 

Program Staff Interview. The third step will involve a detailed interview of 
state agency staff regarding the planning, funding and monitoring 
activities that were carried out in the retraining project. The staff will 
be questioned regarding its understanding of the purpose and 
rationale of the project; its assessment of the retraining plan, 
including the selection of the retraining provider; and its assessment 
of the results of the project. 

Business Interview. The fourth step will involve 2-day interviews at the 
business establishments. The interviews will be conducted with senior 
management and project coordinators, unit supervisors, as well as 
union representatives and trainees. The purpose of these interviews is 
to understand the major business performance goals and skill 
competencies contained in the retraining proposal. The issues 
addressed in these interviews are listed in Table 3 in Appendix A and 
in Appendix B. 
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Identification of Testing Materials. This work task is a review of the content of 
the skill retraining and the identification of comparable skill 
competency instructional and testing material. Businesses or training 
providers thai have used these materials will be interviewed. The 
outcomes of this task will be the basis for assessing the feasibility of a 
company or training provider developing or acquiring similar training 
and testing material in the chosen skill training area. 

Industry and Labor Market Information Review. The final step will involve 
research into the industry and labor market information that state 
agencies may use as the basis for objective industry benchmarks of 
company performance goals and for defining general skill 
competencies and assessing their future labor market demand. This 
will involve a review of available information and. telephone 
interviews with industry association representatives and state LMI 
and economic development staff (See Tables 1 and 2 in Appendix A) 

Business Intervi e ws . 

The business interviews will involve 2-day site visits with separate interviews 
with the chief executive officer of the business establishment (e.g., plant manager), 
the human resource and training managers, the project director or coordinator, 
production supervisors from the units where retrained workers are employed, the 
training providers and retrained workers themselves. The major purpose of these 
interviews is to assess the feasibility of proposed administrative guidelines on 
targeting and substitution and the program outcome measures. Table o lists the 
feasibility topics in each interview session described below. A more detailed 
description of interview topics is provided in Appendix B. 

Session 1: Project Director/Coordinator. The first interview session will be 
held with the project director or the person responsible for the training proposal 
that was developed for the state agency. This session will focus on the approach that 
was taken in designing the project and on its implementation. This session also will 
be used to get an initial introduction to the structure and operations of the 
establishment. Finally, this session will be used to review available written 
information on the company retraining project. 

Session 2: Upper Management. The upper management interviews will focus 
on the company's retraining objectives and on its decision to invest money in 
retraining. In addition, these interviews will focus on the role that the state played 
in the company's final decision to proceed -with the project Thes*, interviews will 
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address specific company performance goals and perceived results from the 
retraining. 



Session 3: Training Provider Interviews. The {raining provider interviews will 
focus on the provider's perception of the purpose and skill requirements of the 
retraining. In addition, the provider will be asked about its role in establishing the 
skill competency requirements and the resources and training time needed for the 
project These interviews will also cover the sources of the instructional, testing and 
evaluation materials use in the retraining project. 

Session 4: Supervisor Interviews. The supervisor interviews will concentrate 
on how the company evaluates the performance of the work unit and what types of 
performance assessment they feel is appropriate after skill retraining and 
competency testing. 

Session 5: Worker and Union Interviews. The final interview session will be 
held with a representative sample of retrained workers and their union 
representatives (where applicable). These sessions will parallel the supervisor 
interviews and will focas on worker testing and selection as well as their perspective 
on work unit performance assessment. 

This first phase of the case studies will result in an interim report. This 
report will discuss the program review and the results of the exploratory case 
studies. It also will include recommendations on revisions to the intervention model 
and to the design of the monitoring and evaluation system. 

Phase II - Final Protect Case Studies 

The second and final phase of the case studies will involve a research design 
that closely follows the research design of the exploratory case studies. The final 
case studies will demonstrate the application of the revised administrative 
guidelines and procedures and program outcome measures to a sample of retraining 
projects. The results drawn from these case studies will be the basis for conclusions 
on the applicability and effectiveness of the proposed evaluation design for state- 
financed, workplace-based retraining programs. These case studies also will be the 
basis for information on each program outcome measure (e.g. company 
performance, wages and earnings, demonstrated job performance) and will serve as 
illustrati *s of the appropriateness and feasibility of the proposed evaluation 
system. F-ally, the case studies will afford the opportunity to review and report on 
alternative net impact designs that may be considered for future implementation by 
these state programs. 
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,„„„! ™ S u f ?- bom phases of ^ P r< * ecl «** suites will contain 
conclnsions on whether: (1) state-financed workplaee-based retraining 
operate m accordance with the policy rationale and intervention model developed in 
tm paper; and (2) me procedural guideline, performance outcomes, ShpS 
meases proposed m tins paper are feasitie and appropriate as the basis for a 
permanent monitoring and evahiation system. This report also will report 
assessments of program and project performance of Phase H case studies based on 
information that is available daring the case stiidy period. These case stSwffl 
provide a preliminary evalnation of state-financed, workplace-based retraining 

E" V* "7* a fc "a*>n of the feasioility of me ^ 
momtonng and evaluation system. 
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APPENDIX A 



Table I 

Proposed Indicators of Program Compliance with 
Business Screening Guidelines and Related Research Questions 



Business Screening Guidelines 


Compliance Indicators 


Data Requirements 


Major Feasibility Questions 


Research Activities 


Selection Guidelines 

Company Performance 
Objectives. 


Percentage of Companies 
presenting measurable 
performance objectives 
in terms of industry 
performance standards. 


Company documenta- 
tion on measurable 
performance 
objectives. 

Company or state 
information on 
industry performance 
benchmarks or 
standards. 


Can companies relate their 
training plans to clear and 
measurable cornpamy perfor- 
mance goals? 

Can companies or state program 
stall identify reliable inform- 
mation on industry benchmarks 
for assessing changes in 
competitive standing? 


Review of training 
proposals. 
Business site inter- 
views. 

Telephone inter- 
views with state in- 
dustry associations. 
Business site inter- 
views. 


major oKill 

Matches. 


Percentage of companies 
documenting major 
changes in skill 
requirements and skill 
mismatches with present 
workforce. 


Company documenta- 
tion of changes in 
skili requirements 
and worker skills. 


Can companies define measur- 
able skill requirements in 
their training plans? 

Can state programs expect 
companies to assess worker 
skills and document skill 
mismatches in the application 
process? 


Review of training 
proposals. 
Business interviews. 


Skill Shortages 
in the State. 


Percentage of compa&ie* 
training in occupational 
areas of recognized skill 
shortages. 


Skill supply and 
and demand data 
for designating skill 
shortages in state 
and local labor 
markets. 


Is there any reliable informa- 
tion on svcjply and demand for 
occupational skills that can 
be used to project or certify 
skill shorages in state or 
local labor markets? 


Review of available 
labor market infor- 
mation for the state. 
Telephone inter- 
views with LMI pro- 
ducers in the state. 


Substitution Guidelines 

State of Need for 
Training Assistance. 


Percentage of companies 
providing written assur- 
ance of need for train- 
ing assistance in order 
to accomplish training 
objectives. 


Company documenta- 
tion in training 
proposal. 


Can state programs establish 
effective financial need 
statements for business 
applications? 


Reviews of training 
proposals. 
Interviews with 
program staff. 
Business interviews. 


Transferable Skill Twining. 


Percentage of companies 
providing general skill 
training that is trans- 
ferable to other 
employers. 


Company documenta- 
tion in training 
proposal. 


Can state programs establish 
reliable guidelines for 
judging skill transfer- 
ability" 


Reviews of training 
proposals. 
Interviews with pro- 
gram staff. 
Business interviews. 



ERIC 



73 

71 



Table I (cont) 



| Business Screening Guidelines 


Compliance Indicators 


Data Requirements 


Major Feasibility Questions 


Research Activities ~j 


Substitution Guidelines front) 
Financial Need. 


Percentage of companies 
providing written docu- 
mentation of financial 
need. 


Company documenta- 
tion in training 
proposal 


What types of financial 
information are necessary to 
establish financial need? 


Reviews of training 
proposals. 
Interviews with 
program staff. 
Business interviews. 


Previous Training Expenditures. 


Percentage of companies 
providing written docu- 
mentation that they have 
not engaged before in 
this type of training at 
the proposed scale and 
scope. 


Company documenta- 
tion in training 
proposal. 


Can companies easily docu- 
ment that they have not 
engaged previously in similar 
training? 


Reviews of training 
proposals. 
Interviews with 
program staff. 
Business interviews. 


Disinvestment Decisions. 


Percentage of *x>mpanies 
providing written docu- 
mentation that the pro- 
posed training is 
necessary to prevent 
company disinvestment. 


Company documenta- 
tion in training 
proposal 


Can compaines easily docu- 
ment that training is 
necessary to prevent company 
disinvestment? 


Review of training 
proposal 
Interviews with 
program staff. 
Business interviews. 
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Table II 

Proposed Measures and Research Issues for 
Assessing Program Outcomes 



Performance Outcomes 


Proposed Measures 


Data Requirements 


Major Feasibility Questions 


Research Activities 


Certifed SHU Competencies. 


Percentage of trainees 
achieving required 
skill competencies. 


Competency tests 
after training 
completion. 


Can companies define measur- 
able skill competencies in 
their training plans? 

What types of competency test 
materials are potentially 
available to companies and 
training providers and what 
are the added administrative 
costs in using similar compe- 
tency materials? 


Review of training 
proposals. 
Business interviews. 

Review of available 
competency 
materials. 
Telephone inter- 
views with users of 
competency 
Materials. 


Work Unit Performance, 


Percentage of work units 
achieving performance 
objectives aftc 3) 
days of cn-the-jcb 
experience. 


Work unit perfor- 
mance data from 
company 90 days 
after training. 


Can companies define 
measurable performance 
objectives for work units in 
their training plans? 


Review of training 
proposals. 
Interviews with 
program staff. 
Interviews with 
union representa- 
tives and trainees on 
evaluating work unit 
performance. 


Company Performance. 


Percentage cf companies 
achieving their perfor- 
mance objectives 1-2 
years after trdning. 


Company perfor- 
mance data reported 
in draining propo- 
sals and follow-up 
reports* 


Can companies relate their 
training plans to clear rind 
measurable company j - r- 
mance goals and set reason- 
able time frames for measuring 
outcomes. 

Can companies be expected to 
collect and provide performance 
information after training? What 
types of confidentiality issues 
will programs likely encounter? 


Review of training 
proposals. 
L >iness site inter- 
views. 


Wages and Earnings of Trainees. 


Percentage of trainees 
maintaing or increas- 
ing annual earnings 
1 year after training. 


Quarterly earnings 
data from state UI 
records fori year 
before and 1 year 
after training. 


Are state UI earnings data 
available on a quarterly basis 
for the required time period? 

What arc the administrative 
costs of acquiring and using 
UI earnings data for program 
evaluation? 


Review of state UI 
earnings data and 
cost information. 

Telephone 
interviews with ES 
staff. 
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Table III 

Indicators of Program Compliance and 
Performance Coutcomes Addressed in the 
Business Interview Sessions 



Administrative Guidelines 
and 

Performance Outcomes 


Session 1: 
Project Director 


Session 2: 
Upper Manage- 
ment 


Session 3: 
Training Provider 


Session 4: 
Supervisor 


Session 5; 
Unions/Trainees 


Selection Guidelines 

Company Performance Objectives. 


Company decision 
to retrain workers. 
Information 
considered in 
setting perfor- 
mance objectives. 
Results from 
training on 
performance 
objectives. 


Comr^ny deciibn 
to retrain workers. 
Information 
considered in 
setting perfor- 
mance objectives. 


Use of per- 
formance 
objectives in 
designing and 
evaluating 
training. 






Major Skill Mismatches. 


Skill testing 
and selection 
of trainees. 




Skill testing 
and selection 
of trainees. 




' Skill testing 
and selection 
of trainees. 


Skill Shortages in the State. 


Company decision 

v\i 1 ci lam vVurr-Cib. 










Substitution Guidelines 

Statement of Need for 
Training Assistance. 


Statement submit- 
ted in training 
proposal. 
Reason for com- 
pany seeking 
training. 


Reason for com- 
pany seeking 
training 
assistance. 








Transferable Skill Training. 


Information 
request on skill 
training require- 
ment-;. 




Information 
request on skill 
training require- 
ments and training 
and testing 
materials. 




Trainee per- 
ceptions of 
skill trans- 
ferability. 
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Table ffl (coat) 





Administrative Guidelines 
and 

Performance Outcomes 


Session 1: 
Project D rec^r 


Session in 
Upper Manage- 
me*it 


Session 3: 
Training Provider 


{ Session* 
Supervisors 


Sessions: 
Unions/Trdnees 




Substitution Guidelines (coat) 
Financial Need. 


Information re- 
quest for financial 
data. 

Comj»any decision 
to seek training 
assistance. 


Company dedsioc 
to seek training 
assistance. 


t 






Previous Training Expenditures. 


Information 
request on 
previous training 
expenditures 
for the 

last five years. 








• 


Disinvestment Decision 


Company decision 
to retrain 
workers. 


Company decision 
to retrain 
workers. 








Performance Outcomes 
Certified Skill Training. 


Information re* 
quest on skill 
training 
requirements. 
Company decision 
on transferable 
skill training. 
Company requests 
for skill testing 
before and/or 
after training. 


Company decision 
on transferable 
skill training. 
Company requests 
for skill testing 
before and/or 
af ten training. 


Information re- 
quest on skill 
training require- 
ments and 
trailing and 
testing 
materials. 






Work Unit Performance. 


Work unit perfor- 
mance exttectA* 
tions of company. 
Company request 
for perfor- 
mance of wcic 
unit after 
training. 


Work unit perfor- 
malice expecta- 
tions of company. 


Information 
request 

on work unit per- 
formance assess- 
ment after 
training. 


Work unit 
performance 
expectations 
of company. 


Work unit 
performance 
expectations 
of company. 
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Table III (cont) 



Administrative Guidelines 
and 

Performance Outcomes 


Session 1: 
Project Director 


Session 2: 
Upper Manage- 
ment 


Session 3: 
Training Provider 


Session 4: 
Supervisors 


Session 5: 
Unions/Trainees 


Performance Outcomes (cont) 
Company Performance. 


Results from train- 
ing on perform- 
ance goals. 


Results from train- 
ing on perform- 
ance goals. 








Wages and Earnings. 


information 
request on 
employment status 
and earnings of 
trainees. 
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APPENDIX B 



DATA REQUESTS AND INTERVIEW TOPICS FOR 
THE EXPLORATORY BUSINESS SITE INTERVIEWS 

The business interviews for the exploratory case studies will involve 2-day site 
visits at the business establishment where the retraining was delivered. These site 
visits will Evolve a review oi data guested in an introductory letter and separate 
interviews with the chief executive officer of the business establishment (e.g., plant 
manager), the human resource and training managers, the project director or 
coordinator, production supervisors from the units where retrained workers are 
employed, and training providers and retiained workers themselves. The proposed 
data requests and interview topics for businesses are outlined below under each 
interview session. 

Business Data Requests 

At least two weeks before the scheduled site visit, businesses will be mailed 
an interview packet that will contain: (1) requests for background data on the 
company and (2) the interview questions for each interview session. 

The background data request will include: 

o Written materials on company structure and product lines. 

o Written materials on retraining plans, instructional and testing 
materials, and evaluations. 

o Information on company retraining expenditures for job retraining 



Topics for Interview Sessions 

Session 1: Project Dhsctor /Coordinator. The first interview session will be 
held with the project director and/or the person responsible for the proposal that 
was developed for the state agency. This session will focus on how the project was 
designed and implemented within the company. This session will also be used to get 
an initial introduction to the structure and operations of the company site. The 
questions in this session will include: 
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(1) What was the major reason that the company decided to restructure its 

process technology or work organization and why was it necessary to 
retrain employees? 

What was the major objective in undertaking these changes? 
How specific were the performance goals that the company set? 

(2) What alternatives were considered when the decision was made to 

retrain current employees or hire new employees? 

(3) Why did the company seek government assistance in worker retraining? 

(4) What was the role of state ret ining funds in your decision to go ahead 

with the changes, including the retraining of workers? 

Did it have any affect on the decision including the time frame in 
which the project was carried out? 

(5) What previous involvement has the company had with other government 

programs (e.g., JTPA, TJTC, economic development programs)? 

How did this involvement affect the decision to participate in the state 
retraining program? 

(6) What was the original timeframe for the retraining project starting from 

original conceptualization, planning and implementation to" post- 
project assessment? 

How did involvement with the state retraining program affect the 
timeframe of the project? 

(7) What were the major issues that were considered in the company 

decision to "make or buy" retraining for employees? 

What criteria were used to choose the most appropriate provider 
including public and private outside vendors? 

What information was used to make the final decision? 

(8) Who actually designed the retraining program and with what types of 

input from management; unions, area educational institutions and 
workers? 

What was the specific skill retraining provided during the project? 

82 



How were the specific skill competencies defined and how were they 
related to job performance issues? 

(9) How were workers actually selected for retraining? 

What factors were considered as to how retrained workers would be 
compensated for their new skills? 

(10) What were the actually costs of the retraining to the company (e.g., 

direct and indirect retraining costs, loss of production/services) and 
how were they different than originally planned? 

(11) How did the company assess the success of the worker retraining 
project? 

What type of evaluation was undertaken by the company? 

What types of evaluation did the company believe should have 
undertaken given the time and resources? 

(12) What were the results of the larger restructuring of which the worker 

retraining was one part? 

What role did retraining play in achieving these results? 
What were some unintended benefits from the retraining? 

(13) How has this retraining project affected company perspectives on 
worker retraining? 

Has there been any change in resource commitment to retraining? 

What lessons did the company learn in doing this type of worker 
retraining? 

(14) What problems would the company foresee in state retraining programs 

requiring businesses to undertake post-training skill testing and to 
report changes in work unit and company performance after 
retraining? 

If this was required, how would this have affected the decision to seek 
retraining funds from the state program? 

Session 2: Upper Man agement The upper management interviews will focus 
why the company decided to invest money in retraining workers and what role 
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the state retraining funds played in making their final decision to proceed with the 
project This interview session will also be used to get a different perspective or 
emphasis on: 

(1) What was the major reason that the company decided to restructure its 

process technology or work organization and why was it necessary to 
retrain employees? 

What was the major objective in undertaking these changes? 
How specific were the performance goals that the company set? 

(2) What alternatives were considered when the decision was made to 

retrain current employees? 

(3) Why did the company seek governmsnt assistance in worker retraining? 

(4) What was the role of state retraining funds in your decision to go ahead 

with the changes, including the retraining of workers? 

Did it have any affect on the decision including the timeframe in 
which the project was carried out? 

(5) What were the results of the larger restructuring of which the worker 

retraining was one part? 

What role did retraining play in achieving these results? 
What were some unintended benefits from the retraining? 

(6) How has this retraining project affected company perspectives on worker 

retraining? 

Has there been any change in resource commitment to retraining? 

What lessons did the company learn in doing this type of worker 
retraining? 

SeSSlQll 3: Training Provider Intervie ws. The training provider interviews will 
be directed at how the purpose of the retraining was perceived by the provider and 
what role the provider played in determining the skill competencies for retraining 
and the resources and retraining time needc*' for the project. 
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(1) "What was the purpose of the retraining project and what particular 

results did the company expect from the worker retraining? 

(2) Who actually designed the retraining program and with what types of 

input from management, unions, area educational institutions and 
workers? 

What was the specific skill retraining provided during the project? 

How were the specific skill competencies defined and hew were they 
related to job performance issues? 

(3) How were workers actually selected for retraining? 

What factors were considered as to how retrained workers would be 
compensated for their upgraded skills? 

(4) What were the actually costs of the retraining to the company (e.g., 

direct and indirect retraining costs, loss of production/services) and 
how were they different than originally planned? 

(5) How did the provider assess the success of the worker retraining project? 

What type of evaluation was required by the company? 

What types of evaluation do you think your company should have 
requested and received given the time and resources? 

Session 4: Supervisor Interviews , The supervisor interviews will concentrate 
on perceived effects of the retraining on productivity improvements in the company 
and how supervisors woUd evaluate improvements in work unit performance. 
These interviews will also address what types of input supervisors had in the design 
and implementation of retraining. 

(1) What involvement if any did the supervisor have in the company 

decision to make changes in technology or work organization? 

What involvement did they have in the design of the retraining and 
the selection of the training provider? 

(2) How were workers selected for the retraining and what factors 

determined who actually c* npleted the retraining program? 
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(3) "What were the major problems and concerns that workers had in going 

through the retraining program? 

(4) How did retraining improve the ability of workers to perform thtlr jobs? 

What clear performance improvements were evident after retraining? 
How long did it take for these improvements to be realized? 

(5) How has this retraining affected the job satisfaction of »r;W • 

workers? 

What other type of benefits were realized through the retraining? 

Session 5: Worker and Union Interviews. The final interview session will be 
held with a representative sample of retrained workers a r J their union 
representatives (where applicable). These sessions will parallel the supervisoi 
interviews and will focus on perceived benefits from retraining and the involvement 
of workers in the planning and delivery of retraining. 

(1) What involvement if any did workers (and union representatives) have in 

the company decis>n to make changes in technology or work 
organization? 

What involvement did workers (and union representatives) have in 
the design of the training and the selection of the training provider? 

(2) How were workers selected for retraining and what factors determined 

who actually completed the retraining program? 

(3) What were the major problems a .d concerns that workers had in going 

through the retraining program? 

(4) How did retraining improve the ability of workers to perform their jobs? 

What clear performance improvements were evident after retraining? 
How long did it take for these improvements to be realized? 

(5) How has this retraining affected, job satisfaction of retrained workers? 

What other type of benefits were realized through the retraining? 
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(6) Do retrained workers now have more job security in the company or 
have marketable skills that could provide opportunities for alternative 
employment with a different employer? 
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